Statistics of natural sensory signals:A key to brain function and efficient signal

processing

Lecture 1: Early auditory and visual systemLecture 2: Statistics in natural imagesLecture 3: Ecological theories of sensory processingLecture 4: Learning of image representationsLecture 5: Learning of sound representations

Lecture 1: Early audition and early vision

ambient air

pressure

• Translating into is an issue.• Psychophysics tries to quantify that translation.• For the sense of touch, the translation is faulty.

air molecules pushed away by string leave a gap and bunch up

low pressurelow density

high pressurehigh density

High & low pressure Sound Wave

Three major points:

These will recur—with more detail—in senses that have been studied more.

HearingWhat is the relevant energy? Sound pressure.

Sound Pressure Level is a measure of force relative to static ambient atmospheric pressure

amplitude

Frequency (# per second)

time

high pressure

ambient

low pressure

Dropped pot complex combo of frequencies

Tuning Fork single frequency

Ul

amplitudefrequencycomplexity

loudnesspitchtimbre

As with touch, experience does not map variables strictly.

•Not 1:1•Different functions for different ƒs. (solid line is 1,000 Hz)

What kind of anatomy underlies such functions?What kind of code does it (can it) use?

Loudness

Amplitude

Consider the ear as a chamber for capturing sound pressure waves. What properties should it have?

•parts that vibrate to transmit the wave

•a cone for collecting sound

outer ear

middle ear

•a cone for collecting sound

outer ear

•some way to note amplitude and frequency

inner earcochlea

Wave travels via bending surface, levers, & fluid.•The inner ear translates to neural.

Uncoil cochlea:

Place of maximal wave depends on frequency of sound: Place Theory

C

Different stiffnesses have different resonant frequencies

Shape of wave envelope depends on amplitude of sound

•increases in thicknessa neural code for properties:

inside is the basilar membrane

•decreases in stiffness•bulged by waves in fluid

Receptors sit on membrane and bend according to place and rate of bulges.

Sound wave properties are copied in neural code. “looks like”

Large Scale Movie

Place of maximal wave depends on frequency of sound: Place Theory: different cells, different pitches

At low frequencies, basilar membrane moves as a unitFrequency Theory: Frequency tone=Frequency impulses

As the basilar membrane moves, the hair cells move andthe hairs are deflected, creating a nerve impulse.

Review:We transduce the presence of a variety of propertiesThe air pressure waves created by vibrating objects

• Gathered by outer ear•Mechanically transferred by Ear Drum, Malleus, Incus, and Stapes •Motion of Basilar Membrane•Deflects hairs, simulates hair cells (receptors)

Two means of coding:Place theoryFrequency theory

Now Vision

Light travels far•we can know about far

objects

First Contact in Vision : energy-->neural impulses-->sensation

• Anatomy suited to properties• Anatomy influences coding—copies properties

surfaces, substances

source

Some light gets to eye

Light: The stimulus for vision

Light travels fast•we can know them

immediately

What properties should the eye

have?

reflected scattered absorbed

Light travels straight•good for image-

production

Image Production

Optical parts

Structures for gathering and focusing light

cornea

iris

lens

pupil optic nerve

What properties should the eye have?

Translating parts

Structures for copying light and sending

signals

fovea

Light scatters in many directions

Some passes through pupil, lens.

Inverted image is projected on the retina.

Lens changes shape to accommodate distance of object to

size of eye.

Image Production

Transducing light

How do we know what’s in the eye, what does what?

Before we had techniques to see cells, we had behavioral data:Go from bright light into dark room—can’t see at first.

Improves for 5 min., levels off… improves again for 15–20 min.

thre

shold

minutes in dark•1st acts fast, adapts less.Kink in function is clue: There are 2 functions•2nd is slow but adapts more.

2 functions 2 jobs 2 types of photoreceptorsLocation, number, sensitivity, connections differ.

So does shape: rods

and cones

•more plentiful•throughout retina

Rods

•fewer in number•fovea only has cones

Cones

[No receptors where optic nerve leaves eye: blind spot gap in image]* Where light hits affects whether it’s noticed

•many:1 with later cellsgreater sensitivity

•1:1 with later cells greater acuity

+

Blind spot:Cover left eye; look at cross with right eye

Fovea and the distribution of acuity

N

VG

C

JP

H

Y

BR

M

S

XZ

X

Q

E

L

T

K

W

F

D

H

•more plentiful•throughout retina

Rods

•fewer in number•fovea only has cones

Cones

[No receptors where optic nerve leaves eye: blind spot gap in image]* Where light hits affects whether it’s noticedTwo types of receptors allow eyes to:•work in dim and bright light•provide sensitivity and clarity•work in B&W and Color

Receptors outnumber cells in the next layer pooling of information, editing, altering before signals are passed along

•many:1 with later cellsgreater sensitivity

•1:1 with later cells greater acuity

Review:Visual system: Contact with light.

Image formationBending light by corneaLimiting by irisFocusing by lensImage on the retinaFovea: acuity

Transduction: 2 receptor typesRods: dim illumination; no colorCones: color vision

Next: The issue of Color Vision

Young-Helmholtz Trichromatic Theory of 19th Century

THREE KINDS OF CONES: “BLUE” “GREEN” or “RED”SHORT MIDDLE LONG

Wavelength ()

MAX ABSORPTION

100%

0%400 500 600 nm

(millionth of a meter)

SENSITIVITY CURVES

QUESTION: YELLOW?

ANY COLOR = some blue, some green, some red

(2) Some mixes of light yield graycomplementary colors:

YR-G & B-Y

(1) Color blindness comes in pairs

Trichromatic theory not the whole story

(3) Color afterimages

Opponent Process Theory: Perhaps outputs of cones are re-coded somewhere into pairs whose members are antagonists (Hurvich & Jameson, 20th Century)

YYB R

G YYB R G

light

optic nerve

fovea

retina

DESIGN OF RETINA

TO OPTIC NERVE

LIGHT

To Brai

n

RODS

CONES

BIPOLARS

GANGLIONS

A NEURAL SYSTEM OF OPPONENT PROCESSES

+

––

CONES

GANGLION

IF + > –, THEN “BLUE”IF – > +, THEN “YELLOW”

IF + > –, THEN “RED”IF – > +, THEN “GREEN”

CONES

GANGLION

–

+ +

FOR BOTH OPPONENT PROCESS SYSTEMS:

IF + = –, THEN “GRAY”

(ACHROMATIC)

Most common color blindness: No red versus green

Brightness Contrast

: Central squares reflect same amount of light.: The darker the surround, the lighter they look.

INTERACTION OF CELLS CODE COLOR, ALSO…

B

A

B

AALEFT looks darker than ARIGHT

Implies interaction in connections between neighboring cells:some signals boosted, some signals reduced

Initial “strength” of signals (registered by rods)

ALEFT = ARIGHTBLEFT > BRIGHT•excitatory or inhibitory

Subsequent connections•end-to-end

•sideways

Signal from BLEFT inhibits signal from ALEFTlateral inhibition

If signal from B exceeds threshold of laterally connecting cells, signal from A will be reduced

Signal from BRIGHT does not affect ARIGHT Consequently, ALEFT < ARIGHT

Mechanism distorts relative to : IllusionIn less contrived circumstances, this same mechanism enhances the detection of an important feature of the world edges

Part I: Solving the problem of

More specific copies or representations are needed.

•Receptors copy properties as best they can (omissions, distortions, errors)•Signals travel to specialized brain mechanisms (broadly: language, space; sensory, motor)

•cortical cells form retinotopic or topographic maps of LVF and RVF•spatially distorted, reflecting importance of receptor region

*receptor signals do not remain separate* many:1 (or a few:1) from receptors to next cells

pooling, editing informationconstruction of representations

Regions of left & right eyes correspond in cortex

sends excitatory signal when stimulated

sends inhibitory signal when stimulated

front view

•many:1 with later cellsgreater sensitivity•1:1 or few:1 with later cellsgreater acuity

side view

Lateral inhibition-type mechanisms

Separate receptors are connected•pool info to ganglion cells•some excite; some inhibit collection is ganglion’s receptive field

Receptive fields “care about” size & shape…

modest rate:stimulus smaller than field

…but not orientation.

”

maximum rate:stimulus “fits” fieldreduced rate:stimulus hits both excitatory and inhibitory cells

But since orientation influences what objects mean Pool some more.

Receptive fields overlap

Across a collection of receptive fields, orientation matters

to cells in the cortex.

They have receptive fields too

Record from 3 cortical cells

simple cells

Some prefer vertical, others prefer horizontal or oblique:

•maximal response to stimuli of a particular orientation ±15°.

response

rate

response

rate

Cortical Cells as Feature Detectors?

rate of firing is only vocabulary

•Provide some where, what, and what’s it doing.

•ambiguity: response is reduced if orientation or size or motion is not exact which is it?

complex cells

Some prefer movement of those features in a particular direction:

•Complex response (to orientation & size & motion direction)

directi

o

n

orientation

response rate

Multiple representations of the retina in cortexin excess of 100,000,000 cells

again

•Hierarchical organization (simple to complex) of visual system builds up ever-better representations

of world.

Form is not meaning, however.

•topography preserves spatial arrangement

•each map extracts some property

ReviewCoding of properties of Visual Stimulation

Color: Trichromatic and Opponent ProcessesContrast: Lateral Inhibition yields Brightness ContrastFeature Detectors of various sortsNot just in CATS! Motion aftereffects

The eye does not send the brain a picture.