From Sensation to Perception - Semantic Scholar · From Sensation to Perception Encyclopedia of Behavioral Neuroscience (Eds. George F. Koob, Michel Le Moal and Richard Thompson),

Shulz, D. and Frégnac, Y. From Sensation to Perception Encyclopedia of Behavioral Neuroscience

(Eds. George F. Koob, Michel Le Moal and Richard Thompson), Elsevier

- p. 1 -

From Sensation to Perception

Daniel E. Shulz and Yves Frégnac CA

Author Affiliations: Unité de Neurosciences Intégratives et Computationnelles (UNIC)

Centre National de la Recherche Scientifique 1 avenue de la Terrasse

91198 Gif sur Yvette, France CA Corresponding Author: [email protected]



- p. 2 -

Synopsis

The scope of this article is restricted to neuronal correlates of perceptual processes localized in the primary sensory cortices of the mammalian brain. These neural structures appear to be a crossroads where feedforward, recurrent, lateral and feedback processing merge together to allow a contextualized perception of peri-personal space. The case-studies that we have chosen illustrate two views: when driven by external sensations and bottom-up activation, the cortical neuronal machinery generates an automatic interpretation of our environment through built-in compositionality mechanisms; when driven by top-down feedback, the same network computes unconscious inferences from sensory events based on predictive knowledge derived from the past. I Feed-forward imprint of the sensory periphery II Beyond the receptive field of sensory neurons: context matters III Sensory illusions as the perceptual expression of stimulus inference by the

brain. IV Study-cases for visual and haptic perception: apparent motion and funneling V. Adaptation of cortical representations to the likely statistics of our “natural”

environment VI Top-down processing and cortical correlates of expectancy



- p. 3 -

Glossary Sensory coding Information in the brain, either evoked by sensory input or autonomously recalled in the form of memories, is represented by changes in neural activity, both excitatory and suppressive (although this latter case is less studied). These changes are expressed in the modulation of the firing rate of single neurons (the “neuronal doctrine”), which in certain cases becomes tuned to specific features of the sensory environment (e.g. feature detectors), or in the spatio-temporal pattern of spiking of activity distributed across neuronal populations (“assembly coding”). “Sparse coding” refers to the special case where stimulus information is encoded by the selective recruitment of a few cells active at irregular points in time. Information theory shows that sparse coding results in an independent component analysis of the sensory scene: it reduces redundancy and maximizes mutual information between the few active units.

Receptive field The receptive field of a neuron is defined by the region of the sensory space within which the presence of a stimulus significantly modifies its activity. In the visual system, the receptive field is the region of the retina where a local change of luminance (relative to the background) modulates (enhances or suppresses) the firing rate or the probability of discharge of the recorded neuron. In the auditory system, receptive fields can be locations or volumes in space, but are also defined as compact regions in the temporal frequency domain. In the somatosensory system, receptive fields are regions of the body surface or internal organs. A special case of tactile receptive fields are those corresponding to the whiskers, specialized hairs which form a discretized mosaic of haptic sensors located on each side of the snout in mammals. Minimal discharge field The minimal discharge field is the part of the receptive field within which the presence of an impulse-like stimulus increases the probability of firing or the number of spikes emitted by a given neuron. This definition excludes the possible recruitment of summation and interaction processes occurring across stimuli in space and time. The surrounding zone, from which only sub-threshold activity is generated (i.e. modulation of the membrane potential or changes in membrane conductance without spike activity), defines the “silent periphery” or “surround” of the receptive field. Hallucinations and Entoptic vision Hallucination refers to a « false » sensory perception in the absence of an actual external stimulus. Various mental episodes can induce the vivid report by the conscious subject of activity patterns witnessing the inner architecture of specific regions of the brain, such as visual cortex. For instance, inner visions can be experienced following retrobulbar blockade of the optic nerves, during acute episodes of aura migraines (« fortifications ») or after taking hallucinogens such as cannabis or mescaline. Clinical analysis of these visual disturbances describes only four constant geometrical patterns, consistently reported by human subjects: (1) tunnels and funnels; (2) spirals, (3) lattices (honeycombs, triangles); (4) cobwebs. As proposed by Christopher W. Tyler, these « entoptic » (vision-from-inside) geometric



- p. 4 -

patterns must be regarded as indicators of some further selective process in the thalamo-cortical pathway, « a kind of functional Golgi stain by which certain neural activities are elevated into consciousness while the majority of possible discharges remain ignored ». Apparent motion The perceptual experience of continuous movement generated by the sequential presentation of two still images flashed in positions which are spatially offset in the visual field. This phenomenon was first studied by Max Wertheimer, the founder of the Gestalt School of Psychology, in his 1912 work: "Experimental Studies on the Seeing of Motion". Mach Bands Mach bands are an optical illusion, named after the physicist and philosopher Ernst Mach. The stimulus is composed of two sectors of uniform luminance, one high (light) and one low (dark), separated by an intermediate zone with a monotonic gradient of luminance which decreases linearly when progressing from the light to the dark sector borders. Humans perceive two narrow “bands” of different brightness that are not present in the physical stimulus. They are situated on both sides of the gradient zone, and are perceived as respectively “brighter” than the light sector and “darker” than the lower luminance region.



- p. 5 -

WHEN FACING A NATURAL VISUAL SCENE, human subjects have an immediate conscious perception of the elementary features that compose it (“segmentation”), as well as of the higher-order global objects that emerge from their associations (“binding”), although not necessarily in this order. Contours, colours, textured surfaces, shapes and tridimensional objects pop-out unambiguously, in a fraction of a second to seconds, according to the background context. For centuries, multiple theories have been proposed to explain the remarkable perceptual capacities of animals and humans in figure-ground segregation and identification of their immediate environment. The structuralist theory of Edward Titchener, dating back to the very beginning of the XXth century, asserted that sensations were the basic elements of perception. In this context, "sensations" were considered as the simplest elementary building blocks open to introspection on the basis of which complex perceptions could be synthesized. In this chapter, the concept of "sensation" has a different meaning than that used by the structuralists. Here we take into account the fact that the central nervous system is immersed in its environment with which it communicates through sensory channels and modalities and on which it exerts actions through specialized effectors. Closer to the enaction theory and to the spike-based computational approaches, we consider "sensations" as the result of the initial processing step corresponding to the transduction of the sensory input detected by the peripheral sensory organs (eye, ear, skin..) into a spatio-temporally formatted spike-based input stream. Through the example of a few sensory illusions, this chapter deals with the long standing question of the link between the stimulus quantified in physical terms and the integrative process realized by the brain before any action in return takes place. Two sequential steps are classically distinguished during low-level perception, the sequence of physiological events in central neural structures, which gives rise to the emergence of a conscious account of the “percept” at the psychological/behavioural level. However, animals and humans cannot be reduced to the status of passive receivers facing an external physical reality, thus other components have to be included, including motivation, attention, expectation, action, decision and memory. These internally generated modulations activate top-down processes which construct unconscious hypotheses about the outer world. We have restrained the scope of this article to neuronal correlates of perception that have been localized in the primary sensory cortices of the mammalian brain: these central networks appear as a major crossroads where feedforward, and recurrent and feedback processing, merge to form a contextualized perception of peri-personal space. The case-studies that we have chosen to illustrate offer examples of situated perception that can be related in neural terms to the Gestalt theory. They support two views which are not mutually exclusive: when driven by external sensations and bottom-up activation, the cortical neuronal machinery generates an automatic interpretation of our environment through built-in compositionality mechanisms; when driven by top-down feedback, the same network computes unconscious inferences from sensory events based on predictive knowledge derived from the past. Consequently, the primal sketch of our sensory periphery is continuously updated and modulated by the feedback or proactive context of our thoughts, intentions and actions. For more than half a century, beginning the pioneering research on the visual system led by David Hubel and Torsten Wiesel, hundreds of laboratories throughout the world have tried to identify the neuronal mechanisms that would explain the



- p. 6 -

remarkable sensory recognition performances of the mammalian brain, never equalled by any artificial device. This research initially targeted cortical primary visual areas, since they form the earliest stage in the functional hierarchy of the visual system where the emergence of high-order feature encoding (orientation, curvature..) and elementary mechanisms for computing geometrical invariants were demonstrated. However what applies to vision generally holds for the other senses. All sensory cortical areas are considered as the seat of memory of our early sensory experience and their functional specialization has been shown to mirror the multimodal perceptual development of the organism, at least during critical periods of postnatal life.

In the case of the visual system, the bi-dimensional distribution of luminance and contrast in the retina is captured by specialized transducer cells, the photoreceptors, where light energy is translated into neural electrical activity. This electrical message is then transferred from retina to subcortical centers including thalamus. The ascending pathway associated with conscious perception is the retino-thalamo-cortical pathway and most of the processing in terms of segmentation and binding is thought to occur at the cortical stage. Three remarkable features of the “early-perceptual” pathway are: 1) the preservation of a topographic representation of space at all intermediate stages of the hierarchy, which provides some form of isomorphism between the sensory image and the feedforward imprint formed at the next step of integration, 2) the existence of multiple loops (feedback from cortex to thalamus, feedback from higher cortical areas to primary sensory cortices) that coexist with the powerful feedforward transmission, and 3) an overwhelming dominance (in terms of relative number of synapses per neurons) of connections (recurrent and lateral) intrinsic to each of the cortical areas, relative to the feedforward contingent. Thus, as soon as a few tens of milliseconds after a stimulus hits the retina (bottom-up processing) the change in neuronal activity, departing from the level of irregular and sparse ongoing activity level, propagates through the nested thalamo-cortical architecture.

These complex activation dynamics are observed not only for exteroceptive sensations arising from the periphery, but also during interoceptive reactivation, for example the recall of a visual memory (top-down processing), or an abnormal imbalance between cortical excitation and inhibition. In this latter case, remarkable illusions, which do not involve attention-related processes, are revealed during migraines and can be amplified by hallucinogenic drugs. It is thought that the paroxysmic activity state of cortex shuts-off the processing of retinal activity and is interpreted by the human observer as an illusory constellation of geometrical patterns. The topological structure of these highly specific “planforms” is closely related to the functional architecture of the cortical analyser. Thus, central neural structures, such as visual cortex, do more than mirroring visual experience. Entoptic hallucinations provide vivid a proof to the observer/patient that his own cortical activity generates a geometrical construct in the absence of sensory input. This construct is interpreted by the mind as a projection of an inner state expressed in retinal-encoded coordinates.

Feed-forward imprint of the sensory periphery The predominant view of sensory processing posits that the filtering properties of the neurons that operate at the different successive steps of the functional integration ladder, from periphery to central cortical areas, become progressively more and more elaborate. This dogma is best illustrated by the hierarchical “simple-complex-hypercomplex” model of David Hubel and Torsten Wiesel and the “grand-mother”



- p. 7 -

concept of Horace Barlow. The critical attributes of a visual scene that trigger electrical activity of neurons in retina, thalamus, primary cortex and higher order cortices indeed show increasing complexity.

While retinal ganglion cells are specialized contrast detectors with a spatially isotropic, opponent (ON vs OFF) concentric organisation, “simple” neurons in the primary cortex show elongated receptive fields with spatially segregated ON and OFF subzones, indicative of the break-down of circular symmetry. Consequently, their firing frequency is tuned for specific features, such as orientation and movement direction. Higher-order visual cortical areas of the dorsal pathway, such as the medio-temporal area, extract information about the motion flow by integrating information over larger areas of the visual space. Even more illustrative of this complexification is the finding of neurons in the inferior-temporal lobe of the ventral pathway that fire action potentials selectively to images of faces, animals or other complex objects. The role of sensory experience in shaping perception is such that there exist neurons in the human medial temporal lobe that are activated selectively when reacting to the picture of famous personalities or even to the sight of their written names. This recognition performance is learned from the social environment of the human observer through different communication channels (radio, TV, movies, speech..) and is dependent on various reward processes. It suggests an explicit code, at least in superior areas of the ventral visual pathway, which might be linked to an abstract representation of iconic memories.

The ultra-fast dynamics observed during pop-out recognition raises a paradoxical challenge in terms of correlation between neural structure and perceptual function. At the behavioural level in humans and monkeys, during a recognition task requiring the discrimination of ‘animal’ versus ‘non-animals’, the response onset delay measured by a forced choice saccadic paradigm can be as fast as one hundred milliseconds (although it is not proven that the motor decision is based on the conscious identification and naming of the animal targeted by the saccade). Taking into account the number of integrative steps through the retino-thalamo-cortical pathway needed to reach prefrontal areas, it has been argued that most of the information processing relies on the first few evoked spikes propagating through a specialized fast pathway. Nonetheless, as we shall see in the next section, it is hard to reconcile an exclusively feedforward processing by the thalamo-cortical pathway with our knowledge of the anatomical connectivity of sensory cortex and in particular the very great number of feedback synaptic contacts.

Beyond the receptive field of sensory neurons: context matters In the primary visual cortex, the integrative properties of visual neurons result from the relative contributions of three sources of afferent connectivity: a) feedforward excitation from the lateral geniculate nucleus, the principal thalamic relay between the retina and the visual cortex, b) feedforward and local feedback inhibition from intracortical interneurons and c) recurrent excitation from other cortical neurons, both within (lateral or horizontal) and outside primary visual cortex, which provides the largest number of synaptic contacts. Geniculate feedforward connections provide visuotopic information to the primary visual cortex that is spatially coextensive with the cortical neuron's minimal discharge field. On average, the extent of this receptive area is relatively small for neurons in the primary visual cortex, spanning only one to four degrees of visual angle in the cat and ten times less in the monkey for central and parafoveal vision. One might then suppose that primary visual cortex neurons only integrate luminance information locally. However, it has been well documented



- p. 8 -

that visual cortical neurons exhibit modulation of responses from far beyond the spatial boundaries of the minimal discharge field. Quantitative analyses of anatomical data in the primary visual cortex of the cat show that axons from the principal neurons of the lateral geniculate nucleus, account for no more than 5% of the total excitatory synapses onto pyramidal neurons of layer IV, the thalamo-recipient layer of the cortex. Consequently, the large majority of excitatory synapses originate from other cortical neurons within the visual cortex and from other cortical areas that project back to the primary visual cortex. It seems unlikely then that the geniculate nucleus alone could fully control the activity of a given cortical neuron, and this seems particularly true for cortical neurons in supra- and infra-granular layers (above and below layer IV). Thus, visual cortex neurons are connected to a widespread network of horizontal and feedback connections that give access to spatially and temporally distributed information over large parts of the visual field.

There is accumulating evidence concerning the functional consequences of this distributed and recurrent connectivity scheme. For example, stimuli outside the minimal discharge field can have potent and complex modulatory influences when presented simultaneously with stimuli inside the receptive field. Neuronal responses in the primary visual areas of the alert macaque monkey to a single oriented line presented in the centre of the neuron's classical receptive field are strongly modulated by the presence of textured patterns of oriented lines presented in the surround of the receptive field. This has been reported most often when the orientation of the surround elements matches that of the centre element but has also been observed, although more rarely, when it does not. The first mechanism is a form of response suppression resulting from the co-linearity of the stimuli whereas the latter one appears to be triggered by the orientation contrast between center and surround.

These observations documented at the cellular level have their counterpart in psychophysical studies where the perception of an object’s attributes depends on the spatial context in which a stimulus is presented. For instance, the finding that primary cortical neurons respond to orientation contrast supports a possible role of primary visual cortex in the mediation of perceptual segregation of texture borders and in perceptual pop-out. Low-level mechanisms can also be implicated in the perception of the apparent lightness of a surface, which is known to be strongly modulated by the spatial context in which it is embedded.

Sensory illusions as the perceptual expression of stimulus inference by the brain.

Tout ce que j'ai reçu jusqu'à présent pour le plus vrai et assuré, je l'ai appris des sens, ou par les sens : or j'ai quelques fois éprouvé que ces sens étaient trompeurs, et il est de la prudence de ne se fier jamais entièrement à ceux qui nous ont une fois trompés. Everything that I have learned so far, to be the most true and reliable, I have learned through my senses : and yet, I have sometimes felt that these senses were misleading, and that it is prudent to never rely entirely on that which has once mislead us.

Méditations Métaphysiques, Première méditation, R. Descartes, 1647.

Hermann von Helmholtz described visual perceptions as unconscious inferences from sensory data and knowledge derived from the past. According to this



- p. 9 -

view, perceptions are regarded as predictive hypotheses made by our brain, which are projected by the mind into the external physical space and accepted as our most immediate reality. The reflexive neural processes to which they correspond find remarkable counterparts in the low-level centre-surround interactions that have been described by electrophysiologists in the primary visual cortex, as well as in the contextual perceptual effects described by experimental psychologists. Some of the best-known illusions, such as Mach bands and apparent motion, open explanatory windows on the inner functioning of the brain, since they result in an apparent contradiction between the physical nature of the stimulus present at the periphery and that reported consciously by the subject (here, luminance “bands” and continuous “motion”). The Mach band was a key to the elucidation of centre-surround antagonism in retinal and higher-order sensory receptive fields. Its signature, in terms of neural correlates, is that of a Mexican hat where focal excitation is surrounded (or opposed) by wide-spread inhibition. Another example is apparent motion, where the percept of continuous motion emerges from sequential static activations. Max Wertheimer, who was one of the founders of the Gestalt School of Psychology, proposed a seminal theory of perceptual grouping in his "Experimental Studies on the Seeing of Motion” (1912). This theory predicts the emergence of coherent percepts of global shape and motion from the temporal staggering of static presentations of elementary spatial features. It assumes the existence of mental processes which favor associations of visual elements in space (according to spatial proximity and similarity in contrast polarity) as well as in time (continuity, common fate). Originally called the “beta phenomenon” by Wertheimer, the apparent-motion illusion is a powerful dynamic effect induced when a visual target is flashed successively, immobile, in different positions in the visual field ordered along a virtual trajectory. Although, at each moment in time, the image that is presented to the observer is stationary, the subject reports the perception of the continuous motion of the object along the trajectory defined by the "association" path which links the various positions explored in succession. The strength of the percept depends on the complexity of the test stimulus, including its shape and texture, on the duration of the static presentations, on the inter-stimulus interval and on the spatial offset between the explored positions.

A series of experimental observations makes it likely that these psychophysical effects are the result of activity waves propagating laterally across the layer plane (which is the projection plane of the periphery) within the primary visual cortex. At the single-unit level, the principal orientation axes of the receptive field of neurons that communicate through long-range horizontal connections are often co-aligned. Horizontal cortical connections are thought to facilitate the response of cells with colinear orientation preference, and to reduce their response otherwise. At a more integrated level, recent local field potential studies show the spontaneous propagation of horizontal travelling waves that most often link cortical loci sharing the same orientation preference. Similarly, at the psychophysical level, numerous studies have described facilitatory or suppressive changes in the ability to detect a central target when adding a contextual periphery. They showed in particular that low-contrast visual contour elements are easier to detect when presented in the context of collinear flankers, confirming that lateral connectivity in visual cortex may participate in establishing such facilitation. Interestingly, the subthreshold synaptic integration field, recorded intracellularly at the cortical level is within the same range as the perceptual ‘association field’ reported by psychophysicists, and thus it is most likely that it provides a cellular substrate for these perceptual effects.



- p. 10 -

Case-studies for visual and haptic perception: apparent motion and funnelling While the spatial contextual effect described above can be easily interpreted in the framework of the “association field", the temporal determinants of this collinear facilitation have been less well explored. The perception of speed can be differentially modulated during apparent motion sequences of oriented stimuli by adjusting the co-linearity and alignment of the local orientation with respect to the motion axis. When observers are asked to discriminate during a forced choice task between the relative speeds of two apparent-motion sequences composed of three identical Gabor patches (an oriented sinusoidal luminance grating whose modulation is weighted by a bi-dimensional Gaussian function) with positional offset, whose orientation is either collinear to the apparent motion axis or cross-oriented to it, a "speed up" illusion is observed. For the same physical speed of the two apparent motion sequences, a Gabor patch moving along its orientation axis appears much faster than a Gabor patch oriented at an angle away from the motion axis.

This psychophysical effect may be quantified by the ratio between the speed of the test sequence and that of the reference sequence for which the subject reports equality in perceived speed. The perceptual bias is as strong as 3-fold in humans, and its strength explains why observers find in the great majority of the cases that collinear sequences are “faster” than non collinear ones even if the two composite stimuli have the same physical speed. The hypothesis that the speeding-up is induced when the horizontal wave travels ahead or in phase with the feedforward inputs from the thalamic stage, is supported both by computational models and by intracellular observations made in cat primary visual cortex for the same stimulus configurations. These different data strongly support the hypothesis of a dynamic neural “association field”: oriented contours should propagate facilitation across space (co-linearity) as well as across time (common fate). In this latter case synergy should be observed when the feed-forward flow travels in phase with the lateral intracortical activity wave evoked by the apparent motion sequence.

The “line motion” effect is another illusion from the same family, induced by asynchronous static presentations. In this latter case, the cue feature is a uniform luminance square, followed by a bar of the same luminance, one end of which encroaches on the previously flashed square. For adequate inter-stimulus intervals and presentation durations, the human subject reports a continuous movement of one border, perceived as the smooth morphing of the square into the elongated bar. The line-motion illusion has been studied using voltage-sensitive-dye imaging of activity propagation, in a secondary visual cortical area of the anaesthetized cat. The aim was to obtain a direct visualization of the spatial spread of the facilitation induced by the cue stimulus. For this purpose, the responses to a flashed small square and a long bar alone were compared with a stimulus configuration producing the line-motion effect, that is a square briefly preceding the bar. In the associative condition, a spreading, low-amplitude activation wave was induced, extending far beyond the retinotopic representation of the initial cue. The observed propagation speed was consistent with the conduction velocity of horizontal axons reported in electrophysiological experiments or inferred with voltage sensitive dye techniques. Furthermore, this work demonstrated that the cue square, even though physically immobile, induced a propagating wave of cortical depolarization, indistinguishable from the spatio-temporal pattern produced by continuous motion of the same square.



- p. 11 -

Although it is likely that similar low-level mechanisms may underlie both the “speeding-up” and “line motion” effects, it should be noted that the temporal parameters that maximize the line motion effect (a delay of 100-200 ms between the presentation of the inducing spot and the flashed bar) are longer than the fast and brief (<100 ms) motion sequences used for the “speeding-up” effect.

Context dependent modulation of cortical activity has been observed in other sensory modalities as well. These modulations and the presence of neuronal signals correlated with the direction of apparent tactile motion have been explored in the whisker-to-barrel cortex system of the rat. Rodents localize objects and discriminate textures by scanning their surface with their facial vibrissae. The exploratory movements of the vibrissae generate spatio-temporally complex sequences of tactile contacts. The sensory whiskers in the mystacial pad of the rat are mapped onto the thalamo-recipient layer of primary somatosensory cortex as distinct units, named "barrels". The detailed description of the organization of the cortical barrel field into discrete architectonic modules has triggered a large number of functional studies in recent decades. In particular, numerous anatomical and extracellular electrophysiological studies have demonstrated a one-to-one correspondence between a mystacial vibrissa and its matching cortical barrel, leading to the concept of a structural as well as functional imprint of the sensory periphery in the cortical representation. Recent observations, using whole cell and intracellular recordings of synaptic responses evoked by individual whisker deflections, have challenged the original notion of a segregated mosaic cortex in the primary somatosensory cortex of adult rats. The cortical spread function, which measures the pattern of divergence of information, is in fact extensive, encompassing several barrels. The spatial extent of subthreshold receptive fields in the vibrissal cortical representation is similar to that we have previously described in visual cortex: it suggests that the rat barrel cortex has a wide array of cortico-cortical horizontal connections that, together with the multi-whisker thalamic input, provide a potential substrate for complex non-linear temporal and spatial interactions. Consequently, context-dependent modulations of responses through the cortico-cortical network might have profound effects on neuronal receptive fields.

Recently, the dynamics of spatio-temporal integration in the somato-sensory cortex (S1) were re-evaluated by using experimental stimuli similar to those encountered naturally during tactile (haptic) exploration. This study took advantage of a newly developed stimulation device that, in the rat, allows the controlled application of large scale spatio-temporal patterns of stimulation, by the parallel and independent activation of the almost complete mysticial pad (stimulation matrix of 25 macrovibrissae). It compared two modes of stimulation: in the first, the vibrissal system was probed with multi-whisker stimuli that were locally invariant and shared the same direction of movement for all whiskers; in the second, the stimulation of several whiskers was coordinated so as to generate an apparent global motion in a given direction. The electrophysiological recordings in S1 cortical neurons showed that the second stimulus (global motion) drove the neurons with a direction preference that could not be predicted from the responses generated from single whiskers when moved independently. These results suggest that individual neurons have the capacity to integrate and extract collective information from the entire whisker pad. As in the visual cortex, we conclude that tactile perception emerges from collective invariants or global properties of the full field input as well as from local independent features.



- p. 12 -

Another famous illusion, still related to lateral intracortical interaction, but expressed with a sensory modality other than vision, is the tactile “funnelling” effect. This is characterized by the percept of spatial mislocalization and increased tactile intensity at a central skin location that is not directly stimulated. When stimulating the skin at three co-aligned points, inputs at lateral sites are “funnelled” centrally so that the perceived intensity at the central site is greater than that perceived when stimulated alone. With two-point stimulation, a funnelled sensation is produced and extends to an unstimulated part of the skin. This illusion has been reported on the forearm, palm, and fingers. A recent functional correlate has been reported in area 3b of the primary somatosensory cortex (S1) of monkeys, where optical imaging showed that simultaneous stimulation of two fingertips produces a single focal cortical activation between the single fingertip activation regions. Thus, in contrast to the traditional view of an isomorphism between the body and its cortical imprint in S1, the topography of the functionally evoked map reflects the “perceived” rather than the “physical” location of the peripheral stimulation. Such contextual influences from beyond the classical receptive field are likely to be determined by mechanisms dependent on intracortical distance, center and surround interactions, and cortical feedback, as described earlier in the case of visual cortex.

Adaptation of cortical sensory processing to the statistical structure of our “natural” environment As beautifully expressed by William James, the founder of the American School of experimental psychology, one hallmark feature of perception is the “constant fit between the mind and the world” in which we dwell. Continuous immersion in a natural environment irregularly updated by motor exploration causes cortical neurons of different modalities to adapt their function to process specific spatiotemporal power spectrum statistics (1/fα). Neural representations of our peri-personal sensory environment in cortex are the result of an ever-active optimization of the fit between brain representations and the physical features of the environment that are experienced during epigenetic development. For instance, during natural vision, informative sensory input is generally gained from the whole visual field and there is now ample evidence that the activity patterns of neurons in the primary visual cortex evoked by full-field exposure to natural scenes differ radically from the activity elicited by simpler and local stimuli. In particular, the most evident qualitative change in response to natural stimuli is the induction of a sparser regime of activity spatially distributed across the cortical network.

Intracellular recordings and voltage-sensitive-dye imaging in the mammalian primary visual cortex show that stimulus-locked variability and network correlations (a measure of information redundancy) both decrease with stimulus complexity, whereas the temporal precision and sparseness of the neural code increase. While drifting gratings evoke highly variable and dense visual responses, stimuli with richer spatio-temporal structures force the cortical network dynamics to become more reproducible at the subthreshold membrane potential level and sparser and more reliable at the spiking level. Thus, the precision of the neural code changes with the complexity of the visual input statistics and their closeness to those of natural scenes. These findings contradict the frequently asserted view that the neural code used to represent sensory information in mammalian visual cortex is noisy and redundant. This apparent low efficiency could be due partly to the choice of visual stimuli used in



- p. 13 -

most previous studies: luminance spots, bars and gratings are typically of low dimension when compared to the high dimensionality of the natural sensory environment, and of limited neuroethological value.

The sparsening of activity produced by natural scenes results from the recruitment of dynamic non-linearities linked mostly to centre-surround interactions. A dramatic contextual reformatting of sensory cortical representations has been reported in awake behaving macaques, when extending the presentation of natural-like scenes beyond the classical discharge field. The concomitant stimulation of the silent surround results in a selective increase in mutual information and spike-based efficiency.

Experimental evidence for sparse coding of natural stimuli has been found for other sensory modalities as well, as demonstrated in the auditory cortex. Here again, stimuli that have high probabilities of being encountered in the natural environment of the animal are optimally encoded by sparse activity. Evidence is much scarcer in the somato-sensory modality, although neural activity in the upper layers of the barrel cortex is particularly low, consistent with the requirement of a sparse regime, where each neuron should be active only rarely. Furthermore, intracortical electrical micro-stimulation experiments, where only one or a very few neurons are selectively recruited, have been reported to bias the perceptual judgement of the observer during a behavioural recognition task in a predictable way. This last set of data strongly suggests that each spike seems to count in the sparse mode regime. Consequently, when our sensory cortices operate in an activation state adapted to the most likely input statistics, any discrete activity change can dramatically influence our perception. This holds, whether the change is produced externally by a sudden discontinuity in the sensory input (that cannot be predicted by our own motor activity) or internally by top-down processes.

Top-down processing and cortical correlates of expectancy We have seen previously that evoked responses in primary sensory cortices can be modulated by concurrent activation of surround regions, beyond the limits of the classical receptive field. We also presented evidence indicative of strong correlates between cortical centre-surround neural non-linearities and biases or illusions in perceptual judgment. While these contextual effects originate partly from the underlying horizontal connectivity intrinsic to the primary sensory cortices, they also depend on the feedback connectivity of higher cortical areas onto the primary sensory cortex. Top-down influences in sensory processing take many different forms, for instance cross-modal interactions produced in synesthesia, attention-related modulation, belief propagation or expectancy. In the latter case, templates of sensory stimuli that occur in a highly reproducible way are continuously updated and constantly compared with incoming sensory information. Detection errors in assessing the presence of a sensory stimulus, or incorrect identification, might arise from the match or mismatch between bottom-up information from the sensory periphery and the top-down feedback arising from stored internal representations and working memory.

Expectancy can be viewed as a dynamic process which prepares the organism to react in an adapted way to sensory inputs which have been primed by the immediate history of the brain. In music cognition, melodic expectation is the tendency for the listener to predict what might come next in the stimulus train, a continuity-in-time feature predicted by the Gestalt theory. Expectancy allows the



- p. 14 -

system to respond -or not- in due time by taking into account the recent presentations of a similar sensory context. It has been studied within two major frameworks. The first, linked to probability theory, posits that in uncertain situations (close to the absolute threshold), responses to highly probable elements are faster and/or more precise than those to rare ones. The second one is more concerned with the impact of the local sensory context and is less directly connected to long-term probabilistic assessments of events. Different dimensions of the stimulus pattern, e.g. temporal, spatial and multimodal can be critical in anticipating the timing as well as the identity of future sensory events.

Expectancy-related physiological signals have been observed in the case of the sudden omission (hence, unexpected absence) of a stimulus during a repetitive sequence of stimulation. This omitted stimulus induces characteristic signatures in human cortical evoked potentials in the visual and auditory modalities. Such a detection of a change within a sequence of temporally discrete events requires computing, representing and retaining for some interval of time the temporal interval between the events together with the event itself. Following this line of argument, the detection of a change or of a violation of expectancy occurs when there is a mismatch between the representation of the current event and the stored representation of the expected event. As illustrated in the case of the cortical mismatch negativity, the characteristic evoked potential associated with such a violation, essentially reflects a pre-attentive, pre-conscious task-independent process that does not require directed attention.

The cortical imprint evoked by new stimuli interacts with spontaneous or attention-gated recall of internal templates stored for the most frequent sensory events. The neural processes through which these bottom-up and top-down representations interact might be supposed to have a distinctive electrophysiological signature. For example a neural response to the expected event should be available to recording during the omission of a stimulus. Such expectancy processes have been studied in the rat vibrissal system. The array of specialized vibrissae is the input stage of an extremely sensitive tactile system, comparable in resolution to the human fingers. Moreover, the natural stimuli in this system are strongly conditioned by the arrangement of whiskers on the mystacial pad that follows a precise geometrical pattern in rostro-caudal rows and dorso-ventral arcs. During the whisking exploratory behavior, rats move their vibrissae rostro-caudally, creating a de facto functional asymmetry between rows and arcs: whiskers in the same row will tend to contact an object successively, whereas whiskers in the same arc either will contact the object nearly simultaneously, or will not contact the object at all. Therefore, the natural whisking movement generates a repetitive pattern of rostro-caudal stimulation that might generate an expectancy wave of activity in the cortex. By defining stimulus conditions as being either predictable or unpredictable, and by recording different neuronal responses across the two types of stimuli, one can infer whether expectancies of the organism account for the observed differential responses. A clear expectancy response can be detected in humans by measuring auditory evoked potentials in humans during repetitive stimulation with a low rate of omission (around 10%). In the somato-sensory barrel field, omitted stimuli protocols reveal neural correlates which can be recorded even in anaesthetized animals, consistent with the pre-attentive nature of the mismatch negativity reported in the auditory cortex. In conclusion, these results are in agreement with the idea that somato-sensory cortex is generating hypotheses about stimulus characteristics on the basis of stored



- p. 15 -

representations which are compared with the ever updated information stream incoming from the sensory periphery.

Other forms of expectancy are used by the brain to filter out the sensory perturbation produced by the self-generated motor activity which serves to gather relevant sensory information. The best known examples have been described in the visuo-oculomotor system in mammals and in the electrosensory system in the electric fish. Both provide examples, at the functional and structural level, of the existence within the brain of internal copies (outflow, efferent copy) of motor signals (saccades or electric discharge). These contextual signals prepare the sensorium to ignore the expected recalibration of the sensory space produced by the sudden shift in gaze fixation or the generated electric field.

From this overview, we conclude that low-level perception can be defined as a cortical-based computation, constantly building unconscious or self-generated inferences during the processing of sensory events. The outcome of the perceptual process is highly conditional on the context of predictive knowledge derived from the past sensorimotor experience. In many cases, perception departs from the physical reality, but this is what ultimately feeds and guides our interactions with the world. Acknowledgments We thank Drs Kirsty Grant and Andrew Davison for helpful comments. Supported by CNRS, ANR (NATACS, NATSTATS), and the EC FET-BioI3 initiative: FP6-015879 (FACETS)). See Also the Following Articles Disorders of face processing, Brain imaging, Consciousness, Temporal lobe and neural mechanisms, Novelty, Attention and speed of information processing, Cortical visual processing, Mirror-neuron system, The role of neuronal synchrony in normal and pathological brain Further Reading Chen, L.M., Friedman, R.M., Roe, A.W. (2003) Optical Imaging of a Tactile Illusion in Area

3b of the Primary Somatosensory Cortex, Science 302, 881-885.

Barlow, H. (1972). Single units and sensation : a neuron doctrine for perceptual psychology ? Perception 1(4) 371-394.

Bringuier, V ., Chavane, F., Glaeser, L. and Frégnac, Y. (1999). Horizontal propagation of visual activity in the synaptic integration field of area 17 neurons. Science. 283 : 695-699.

Delorme, A. Flückiger, M. (2003) Perception et réalité: Introduction à la psychologie des perceptions. Paris: De Boeck Université.

Frégnac, Y. (2003). Neurogeometry and entoptic visions of the functional architecture of the brain. Journal of Physiology - Paris 97 87–92.

Frégnac, Y., Baudot, P., Chavane, F., Lorenceau, J., Marre, O., Monier, C., Pananceau, M., Carelli, P. and Sadoc, G. (in press). Multiscale functional imaging in V1 and cortical correlates of apparent motion. In “Dynamics of visual motion processing”, Editors (G. Masson and U. Ilg), Springer.



- p. 16 -

Gibson J.J. (1979) The ecological approach to visual perception. Hillsdale, New Jersey: Lawrence Eribaum Associates, Inc.

Gilbert, C.D., Sigman, M. (2007). Brain States: Top-down influences in sensory processing, Neuron 54, 677-696.

Hube, D. and Wiesel,T.N. (1982). Eye, Brain and Vision. Scientific American Editions.

Gregory, R.L. (1997) Knowledge in perception and illusion. Philosophical Transactions Royal Society London Series B: 353, 1121-1128.

Jacob, V, LeCam, J., Ego-Stengel, V. and Shulz, D. (2009). Emergent properties of tactile scenes selectively activate barrel cortex neurons. Neuron. 60(6): 1112-1125.

Jancke, D., Chavane, F., Naaman, S., Grinvald, A. (2004) Imaging cortical correlates of illusion in early visual cortex, Nature 428, 423-426.

Olshausen, B.A., Field, D.J. (2005) How close are we to understanding V1? Neural Comput 17, 1665–1699.

Ratliff, F. (1965). Mach Bands, Holden-Day, San Francisco

Seriès, P., Lorenceau, J. and Frégnac, Y. (2003). The "silent" surround of V1 receptive fields: Theory and experiments. J. Physiol. (Paris). 97: 453-474.

Thorpe, S., Fize, D. and Marlot, C. (1996). Speed of processing in the visual system. Nature. 381 : 520-522.

Varela, F.J., Thompson, E., Rosch, E. (1991) The Embodied Mind: Cognitive Science and Human Experience, Cambridge, MA: The MIT Press.

Cross-references :

210: Cortical visual processing

105: Mechanisms for the attentional modulation of visual processing* 152: Neural Plasticity in Visual Cortex 45: Hallucinations 53: Consciousness/ unconscious 216: Peripersonal space and body schema



- p. 17 -

Figure : Sensation, perception and illusion A – Mach Bands : from top to bottom, displayed luminance profile, measured luminance gradient in positions 1 to 4, perceived luminance profile. The Mach illusion (blue arrows) corresponds to the perceived lighter (position 2) and darker (position 3) bands.

B – Line Motion Illusion : top, spatio-temporal sequence of displayed stimuli (square followed by a bar) ; bottom , perceived downwards motion (blue arrow).

C – A conceptual framework (inspired from Richard Gregory).

Documents

From Sensation to Perception - Semantic Scholar · From Sensation to Perception Encyclopedia of Behavioral Neuroscience (Eds. George F. Koob, Michel Le Moal and Richard Thompson),