Lecture 5bCSE3030: GUI Technology1 Human Perception: Vision, Language and Some Memory Dan Eaves CSE3030 Lecture 5b

Lecture 5b CSE3030: GUI Technology 1

Human Perception:Vision, Language

and Some Memory

Dan Eaves

CSE3030

Lecture 5b


Vision

• The primary tool for interaction is vision.• We need to know something about how the

eye/brain system works.

• We can start by looking at some optical illusions (all these and more can be downloaded from the web site.)

• The critical point about optical illusions is that:

• You can’t not see them, even when you know they aren’t there!


The Necker Cube

Is the ball inside or out? (You can’t see it both ways at once, it “pops”.)


Hummm

• Do the black dots actually exist?


• Spirals or circles?


Too Much Scotch

• Straight or bent?


Move your head

• What’s moving?


Perspective Usually Helps

• Who is bigger?


How Many Colours?

• Just 3!


Virtual Moire

• This reproduces the feeling of the moire effects we need to avoid: http://www.mathematik.com/Moire/

http://www.mathematik.com/Moire/




Your Eyes Are Part of Your Brain(Well, the retina is)


Another View


A Schematic View

• Weird– The left half from both eyes goes to the right half of the

brain, and the reverse.

– (Each brain hemisphere runs the other side of the body).


Inside your eye

• The area in the back with the upside down image is the retina.


The Retina

• The fovea sees things in detail• The macula (peripheral vision) mainly perceives

movement (tiger detector)• The fovea covers 2 degrees well, and another 2-3

not so well – a tiny part of what you think you see!


What you actually see in focus, 1

• Shows what your eye actually sees when you are reading a book, not what it feels like you see.

• Your foveal system uses roughly 50% of the visual nerves for transmitting the information of the 100% acuity spot (approx. 1/10'000th of your total visual field).


What you actually see in focus, 2

• Look at top left


How does your brain fake it?

• At all times your eyes are jumping around, saccades, every 20+ milliseconds

• There’s a sort of image of your field of view held in the visual cortex at the back of your brain, which is what you’re conscious of.

• Your peripheral vision detects motion.– Blinking is perceived as motion– You cannot resist looking at moving things– Blinking is bad!


A Last Illusion

• For some really good interactive experiments and effects, go to:

• http://psylux.psych.tu-dresden.de/i1/kaw/diverses%20Material/www.illusionworks.com/index.html

Here is some text


Other Vision Problems

• Optical Ageism– Consider bifocal glasses.

– Consider the traditional position of the monitor, on top of the computer.

– The combination will be a rapidly stiffening neck.

• Personal taste and flash design– You may think it’s beautiful.

– (Your mother may think it’s beautiful…)

– But some percent of your users may hate it

– Drab is best for business applications, and restraint is better for Web applications


You can’t see well if your middle ear is destroyed!

• How is it that what you see looks steady while you are moving?

• The visual cortex is wired into the middle ear’s balancing system and reconstructs what you see accordingly.

• No middle ear = blurred sight when moving your head (like walking).


A Good Approach?


Remember, the left hemisphere controls the right half of the body

• Why?• Language is a specialisation of the left side of the

brain– Almost all right handed people

– Most left handed people

– (Why are 10-15% of people left handed?????)

• Remember, language has been with us for a (relatively) long time (100k years? A million?)

• Writing is brand new and still strange.


Your brain

• Notice how small the connection between the two hemispheres is – that’s the corpus callosum

• (If it’s cut in half, are there two of you?)

Corpus Callosum


What’s in there

• About a kilogram of stuff– 2% of body weight

– 20+% of metabolic activity (having a big brain is expensive!)

• Maybe 1011 neurons• Maybe 10,000 connections (synapses) per neuron.• Something kind of like 1015 bits!• The most complicated object in the known

universe (with the possible exception of the world’s telephone system considered as a single entity).


Our brain is wired to understand and produce speech

• Broca’s area is close to the mouth, Wernicke’s is close to the (left) ear. Their functions match their locations.

• (We know their functions because lesions (injuries) in these areas cause different problems.


Brain of Leborgne(“Tom”, who Broca treated in the 1860s)

• A stroke destroyed his Broca’s area


Damage to Broca’s Area

• Comprehension still rather good.• Word generation hard, difficulties in getting started.• Great frustration; aware of defect.• Frequently, problems with “the little words”: of, to,

and, the, with, for, on, is, by, was ...– These are completely abstract

• John used the pencil to write his name… understood• The pencil was used by John to write his name… not

understood


Damage to Wernicke’s Area

• Little comprehension• Extends to what is said as well as what’s heard.• “Word salad”• Absence of frustration: doesn’t know has problem

• There are really good examples of both posted on the web site.


Sign Language

• A visual language invented by the deaf• Not spelling out words, not (necessarily) related to

local spoken language (ASL -- American Sign Language -- related to French, not English)

• You communicate in a 3-D space around you.• Typically, visual processing happens in the right

brain, language processing in the left.• Sign language processing happens in the left, and

damage to Broca’s and Wernicke’s areas have same effects!


Creoles

• Language invented by the children of people who speak a pigeon.

• Pigeon: in a community made up of people from a lot of language backgrounds, communication takes place using a collection of words from different languages: no grammar.

• (Pigeon usually happen where lots of agricultural workers are brought in from different countries in semi-slavery to work together: Hawaii, New Guinea, Haiti, etc. But note, New Guinea pigeon is actually a creole.


Creoles 2 -- South Africa

• FANAKALO (Miner’s Language) – Pidgin, Zulu based, around mines.

• OORLAMS -- Creole, Afrikaans based. • TSOTSITAAL (Fly Taal; Flaaitaal) -- Creole,

Afrikaans based, can converse with Afrikans. • ISCAMTHO -- Creole, intentionally not Afrikaans

based, private with Afrikaans excluded.• For more information: • http://www.ethnologue.com/show_country.asp?name=South+Africa

http://www.ethnologue.com/show_country.asp?name=South+Africa





Creoles, 3

• Creoles are invented by the children in a pigeon-speaking community.

• All creoles studied exhibit the same gramatical structures… Which are much more elaborate that those of any established language.

• Example: verb tenses to distinguish regular versus occasional versus one-time action; action planned versus action completed; etc.

• Is what these kids invent the “real” human hardwired grammar???


“Advanced” Languages

• Over time, languages seem to evolve by becoming simpler.

• E.g., French has noun endings to indicate use, gender, etc.

• English doesn’t have these, but does still have verb tenses… but not all of them, e.g., no formal subjunctive.

• Chinese has neither noun endings or verb tenses, doesn’t have articles (a, an, the) either

• A good case can be made for saying that Chinese is a more advanced language than English!


But Chinese Is Tonal• (Well, there’s one written Chinese language, but

there’s no single spoken one: Mandarin, Wu, Xiang, Cantonese, Min, Hakka, and Gan.)

• “dim sum” from the Cantonese for little hearts; the same characters are pronounced "dian xin" in Mandarin (“yum cha” is Cantonese for drink tea).

• These vary in the number to tones used:– Mandarin = 4; Hakka = 6; Cantonese = 9

• 天唔驚，地唔驚，只驚北方人講廣東話唔正 !


Which Chinese Characters?• That quote is written in traditional Chinese characters.• In China, simplified Chinese characters are used, so the

quote can’t be read there.• But in many centres of overseas Chinese settlement,

traditional characters are used – but not in Singapore or Malaysia!

• Further, there are two+ technical computing vocabularies, developed – In China during the period of isolation (1950-1970)

– Outside of China during the same period

• Handling documentation in Chinese is not easy!


Your Memories

• Sensitory Memory– One per sensitory modality: sound, vision, etc.– Decay in .7-2 second

• Working Memory– Capacity of 7 2 chunks– Decay in 5-7 seconds unless rehersal

• Long Term Memory– Infinite capacity(???)


It is not known

– How the “cocktail party syndrome” works

– Where the working memory is stored

– How long term memories are formed

– Where and how long term memories are stored

– What’s in memory: Pictures? Words? Something else?

– How chunks are formed

– How long term memory is indexed

– Whether we ever really forget or simply corrupt the index

– How (given the 3 part model) we remember where the car is parked


We do know

– The hippocampus is involved in forming long-term memories

– The hippocampus itself is probably a mid-term memory (where we remember where the car is parked)


Another View

– The view from below, with and without the material in the way.


The Two Types of Long Term Memory

• Know how versus know what– Know how = skilled knowledge from great experience

– Know what = book learning

• Access to the two sorts of knowledge is different– Know what = conscious recall -- What is the capital of

Australia?

– Know what = unconscious action, you find yourself doing the right thing -- When do I shift into second? What kind of loop shall I use here?

• Know how develops from know what with experience


Another View of the Long Term Memories


Anderson’s ACT* Model

– (There should be a big arrow from the Declarative to the Procedural memory!)


Nobody Knows

• If any of these models are realistic.• What a memory looks like and how it is stored• How skilled action is performed• But, using various newer techniques, CAT, PET,

MRI, fMRI, etc, can begin to see what brain areas are active as certain tasks are performed.

• (And several couples have recently made love in such machines, so we know a little about that, too.)


Examples

• Whoops...

Documents

Lecture 5bCSE3030: GUI Technology1 Human Perception: Vision, Language and Some Memory Dan Eaves CSE3030 Lecture 5b