Vidna kognicija IV

Vidna kognicija IV

Danko Nikolić

Sinestezija

Sinestezija

Grčki: (syn) ujedinjenje + (aesthesis) osjeti

“Ujedinjenje osjetila”

Ramachandran &Hubbard, 2003.

Graphemes -> colors = 517/778 = 66.5% Smells -> temperatures = 1/778 = 0.1%

Time units -> colors = 177/778 = 22.8% Smells -> touch = 5/778 = 0.6%

Musical sounds -> colors = 144/778 = 18.5% Sounds -> kinetics = 4/778 = 0.5%

General sounds -> colors = 113/778 = 14.5% Sounds -> smells = 14/778 = 1.8%

Phonemes -> colors = 77/778 = 9.9% Sound -> tastes = 48/778 = 6.2%

Musical notes -> colors = 75/778 = 9.6% Sound -> temperatures = 4/778 = 0.5%

Smells -> colors = 53/778 = 6.8% Sound -> touch = 31/778 = 4.0%

Tastes -> colors = 51/778 = 6.6% Tastes -> sounds = 1/778 = 0.1%

Pain -> colors = 45/778 = 5.8% Tastes -> temperatures = 1/778 = 0.1%

Personalities -> colors = 43/778 = 5.5% Tastes -> touch = 4/778 = 0.5%

Touch -> colors = 31/778 = 4.0% Temperatures -> sounds = 1/778 = 0.1%

Temperatures -> colors = 19/778 = 2.4% Touch -> smell = 2/778 = 0.3%

Orgasm -> colors = 8/778 = 1.0% Touch -> sounds = 3/778 = 0.4%

Emotions -> colors = 8/778 = 1.0% Touch -> tastes = 5/778 = 0.6%

Kinetics -> sounds = 3/778 = 0.4% Touch -> temperatures = 1/778 = 0.1%

Musical notes -> tastes = 1/778 = 0.1% Vision -> smells = 8/778 = 1.0%

Personalities -> smells = 3/778 = 0.4% Vision -> sounds = 12/778 = 1.5%

Personalities -> touch = 1/778 = 0.1% Vision -> tastes = 16/778 = 2.1%

Smells -> sounds = 4/778 = 0.5% Vision -> Temperatures = 2/778 = 0.3%

Smells -> tastes = 1/778 = 0.1% Vision -> touch = 8/778 = 1.0%

Osnovne karakteristike sinestezije

• Osjetna iskustva izazvana neuobičajenim podražajem

• Živo i detaljno perceptivno iskustvo• Automatska asociacija; Nema voljne kontrole• Od ranog djetinjstva (“otkad znaju za sebe”)• Asocijacije se (uglavnom) ne mijenjaju

vremenom

Poznate osobe sinestete

• Wassily Kandinsky

• Vladimir Nabokov

• Franz Liszt

• Nikolai Rimsky-Korsakov

• Nikola Tesla

• Richard Feynman

Sensorno iskustvo

Induktor Dodatno iskustvo

Sensorno iskustvo

Drugačiji model

Semantička reprezent.

Induktor

Dodatno iskustvo

Sensorno iskustvo

Sensorno iskustvo

Test for eccentricities:1 Report that Synaesthesia disappeared at higer eccentricities: An indication for a perceptual aspect of synaesthesia

We retested our synaesthetes: none showed a dissapearence of synaesthesia at higheccentricities

All our synaesthetes reported, that letters and digits do change color depending on theirmeaning, e.g, O and 0.So we tested our synaesthetes with

-3 Homographs: -Montage (Installation)/Montage (Wochentage)-Collagen (Plural von Collage)/Collagen (Bindemittel)-Heroin (Suchtmittel)/Heroin (Heldin)

Dixon et al 2000

-one synaesthete vs. control-arithmetic test, where the concept of a digit can inducesynaesthesia

Rich an Mattingley 2003 Ambiguous stimuli

-14 synaesthetes vs. 14 controls-first, a stroop test was performed-next, they asked the subjects to attend to either the globalor the local stimuli only-=> the interference effect was significantly reduced, comparedto the first experiment

Palmeri et al., PNAS, 2002

However, Edquist et al. (2006) could not replicate these results on a larger group of synaesthetes.

[Oben sehen Sie eine Studie aus dem Jahr 2002. Hier war die Aufgabe, in dem schwarz-weiss Bild das Ziel, in diesem Fall die zwei, zu finden. Da der Proband das Ziel farbig sah (rechtes Bild) war er schneller, als die Vergleichsgruppe (sog. “pop out” Effekt). Dieses Ergebnis konnte allerdings 2006 nicht wiederholt werden.]

Serial visual search

Set size

Tim

e

Pop out

Serial search

Shape search task

-same 2 synaesthestes-40 controls-Display: 5´s scattered randomly on a white screen, amongthese 2´s forming a global shape like a triangle (4 shapes)- each stimulus was presented for 1 s

Result: The synaesthetes were significantly (81.25%)better than the control (59,4%)

A crowding study

Stroop zadatak

Stroop, 1935

ZelenaZelenakongruentna nije kongruentna

CAR CAR

• Result: Naming the congruent colour is faster than naming the incongruent colour

• No effect with words without a colour meaning:

• Indicates automatic and involuntary processing

Stroop task:

Grün, Rot, Blau, Gelb

vs.

Grün, Rot, Blau, Gelb

Conclusion: reading is an automatic process

Stroop u sinesteziji

• Zadatak: “Imenuj stvarnu boju slova.”

• Resultat: Sinestete imenuju pravu boju slova brže ako ta boja odgovara sinestetskoj boji (kongruentne boje) nego ako se te dvije boje razlikuju (ne kongruentne) (e.g., Odgaard et al.,

1999).

• Result: Synaesthetes are faster in naming the true color if this color is the same as (congruent to) synaesthetic than if these colors are different (incongruent) (e.g., Odgaard et al.,

1999).

• Tipični Stroop zadatak ne dokazuje da synaesthesia proizvodi stvarna perceptualna iskustva.

• Npr. Stroop efekt se može očekivati čak i za obične asociacije, “Znam da je limun žute boje”.

Međutim:

Sensorno iskustvo


Sensorno iskustvo

Studija I: Priroda dodatnih iskustava

Colour-opponency in synaesthetic experiences

Activation in colour-specific areas

• fMRI provided evidence for an activation of the colour specific areas V4/V8 of the visual cortex during synaesthetic color experience. (e.g., Hubbard et al., 2005; Sperling et al., 2006)

• This suggests the involvement of early stages of visual processing

Colour opponency Theory

• First stage: Receptor stage consists of the three photopigments (blue, green and red cones).

• Second stage: neural processing stage where the colour opponency occurs.

http://en.wikipedia.org/wiki/Image:Opponent_colors.svg

Krug boja:

Sinestetska boja

Oponentna boja

Nezavisna boja

Color-opponency in synesthesia• Test color opponency in synaestetic experiences by Stroop

task

+-

+-

or

• Stroop effect can occur for non-synaesthetic associations. For example, “I know that lemon has yellow color”.

• Typical Stroop task is not a direct proof that synaesthesia produces a real perceptual-like experience.

Hipoteze• Grafem-boja sinestezija

-

-Sem. Real

Oponentne boje

Sem. Real

-

-

Ne oponentne boje

+

+Syn. Real

Iste boje

• Semantičke asociacije (npr. žuti limun)

-

-Syn. Real

Oponentne boje

Syn. Real

Ne oponentne boje

Congruentwith synesthetic (C)

Incongruent opponent (O)

Dizajn:

C I O B

Design

• 6 synaesthetes: 2 projectors, 4 associators

• 5 graphemes per participant, presented in three colours plus baseline condition

• Each grapheme was presented 10 times; a total of 200 trials

• 2 control participants per synaesthete, matched by gender and age

C G :

IH :

N B :

8 A E O U

U I E A 3

3 5 9 M A

K L :

T Q :

K T :

4 A E 3 U

A E I O U

2 5 7 E O

Six synaesthetes:

Distribution of colors

ResultsR

esp

on

seT

ime

[ms]

Stimulation condition

650

700

750

800

850

900

950

1000

1050

1100

C I O B

Control subjects(n=12)

Res

po

nse

Tim

e[m

s]

Stimu lation condition

650

700

750

800

850

900

950

1000

1050

1100

C I O B

Synaesthetes(n=6)

p=0.89 p=0.005

C I O B C I O B C I O B C I O B C I O B C I O B

Stimu lation condition (individual)

6 00

7 00

8 00

9 00

1 00 0

110 0

1 20 0

(C G ) (T Q ) (IH ) (N B) (KL ) (KT )

Individual synaesthete subjects

Stimu lation cond ition (grapheme)

IH :

Individual graphemes for one subject

Conclusions

• Reaction times for incongruent colours were overall slower than the congruent condition

• Reaction times for the congruent colour showed a facilitation

• Strong indication that synaesthesia arises within colour-opponency

Congruent(C)

Incongruentopponen t (O)

Incongruentindependen t (I)

Kontrolni eksperiment

Semantičke veze:

Tri objekta:

Res

po

nse

Tim

e[m

s]

C CI IO O

ResultsControl subjects

(n=8)Synaesthetes

(n=4)

~50 ms

p=0.19

Res

po

nse

Tim

e[m

s]

Stimulation condition

650

700

750

800

850

900

950

1000

1050

1100

C I O B

Res

po

nse

Tim

e[m

s]Stimu lation condition

650

700

750

800

850

900

950

1000

1050

1100

C I O B

Res

po

nse

Tim

e[m

s]

C C CI I IO O O

IH :

Individual results for one subject

Conclusions• In grapheme color synaesthesia, the interference in

the Stroop task has two components: – i) semantic component and – ii) color-opponent component.

Nikolić, Lichti & Singer, Psychological Science (2007)

• This indicates early visual representation of synaesthetic color experiences.

• Colour-opponency Stroop task is an objective method to test whether someone is a synaesthete.

Sensorno iskustvo


Sensorno iskustvo

Zaključak

Colour opponency

task

Second study


[Zweite Studie. Ein Studie über serielle visuelle Suche in der Synästhesie.]

Sensorno iskustvo


Sensorno iskustvo

Studija II: Sensorna priroda induktora


Set size

Tim

e

Pop out

Serial search

Palmeri et al., PNAS, 2002

One synaesthete:

However, Edquist et al. (2006) could not replicate these results on a larger group of synaesthetes.

Larger sample than 1

• Six synaesthetes with grapheme-colour synaesthesia. • Synaesthesia proven by colour opponency task. • Also, highly consistent selection of colours over several

month period. • Four ‘associators’ and two ‘projectors’.• No “pre-selection” procedure. Everyone who responded to

our advertisement was taken into the study. The sample represents the population.

• Every synaesthete had different colours associated to digits 2 and 5.

• Two control subjects per synaesthete, matched by gender and age (±1 year).

Hypothesis 1: Only projectors can search fast

Hypothesis 2: Need to be aware that shapes represent digits

Two sessions:1st) Without instruction: No mention of digit shape; No instruction to use synaesthesia.

--half an hour break--

2nd) With instruction: subjects informed that shapes indicate digits and are asked to take advantage of synaesthetic colour associations.

Hypothesis 3: Fast processing of distractors

Familiar shapes increase the speed of normal visual search (i.e., without synaesthesia); Malinowski & Hübner (2001).

Res

pons

e ti

me

(ms)

Ciljni objekt Distraktor

Naša studija:

- Veći broj različitih podražaja:

Type of stimuli

×

Digit instruction

×

Synaesthesia

×

Target

5×2×2×2

Repeated measure design

Sample• The same six synaesthetes • No “pre-selection” procedure. Everyone who

responded to our advertisement was taken into the study. The sample represents the population.

• Highly consistent selection of colours over several month period.

• Four ‘associators’ and two ‘projectors’.• Two matched control subjects per synaesthete.

Results

Target-present synaesthetes

-500

500

1500

2500

3500

4500

0 10 20 30 40

Size

RT [m

s]

T D

0

20

40

60

80

100

120

1 2 3 4 5

Slo

pe

[ms/

elem

ent]

Synaesthetes

Controls

Target present

0

10

20

30

40

50

60

70

Synaesthetes Controls

Slo

pe

(ms/

elem

ent)

Overall average:

48

50

52

54

56

58

60

62

64

CG Controls

0

10

20

30

40

50

60

70

80

90

100

KT Controls

0

5

10

15

20

25

30

35

40

45

TQ Controls

0

5

10

15

20

25

30

35

40

45

50

KL Controls

0

10

20

30

40

50

60

70

IH Controls

54

56

58

60

62

64

66

NB Controls

Slo

pe (

ms/

elem

ent)

Slo

pe (

ms/

elem

ent)

Projectors Associators

0

20

40

60

80

100

120

140

2,5 rot M,2 2,5 4, inv4 inv4, 4

CGControl

0

20

40

60

80

100

120

140

2,5 rot M,2 2,5 4, inv4 inv4, 4

KTControl

0

10

20

30

40

50

60

70

80

90

2,5 rot M,2 2,5 4, inv4 inv4, 4

TQControl

0

20

40

60

80

100

120

2,5 rot M,2 2,5 4, inv4 inv4, 4

KLControl

0

20

40

60

80

100

120

2,5 rot M,2 2,5 4, inv4 inv4, 4

IHControl

0

20

40

60

80

100

120

140

2,5 rot M,2 2,5 4, inv4 inv4, 4

NBControl

Projectors Associators

Slo

pe (

ms/

elem

ent)

Slo

pe (

ms/

elem

ent)

0

5 0

1 0 0

1 5 0

2 0 0

2 5 0

3 0 0

C G

C o n tro ls

0

20

40

60

80

100

120

140

C G

C ontro ls

Target present Target absent

0

10

20

30

40

50

60

70

80

Session1 Session2

Slo

pes

(m

s/el

emen

t)

SynaesthetesControls

Instruction about shapes representing letters

Synaesthetes

0

20

40

60

80

100

120

2,5 rot M,2 2,5 4, inv4 inv4, 4

Session1Session2

Controls

0

20

40

60

80

100

120

2,5 rot M,2 2,5 4, inv4 inv4, 4

Session1

Session2

Slo

pe (

ms/

elem

ent)

Digit instruction, details:

Conclusions I

• Synaesthetes do not process distractors faster than do non-synaesthetes.

• An explicit instruction to use synaesthesia did not improve performance.

• There is nothing special about projectors regarding serial search.

We had to reject all three hypotheses:

Conclusions II

• Synaesthetes do not perceive perceptual ‘pop-out’ in serial search tasks.

• Synaesthetes are not, in any way, faster than non-synaesthetes in this task.

• Fast serial search is not typical of synaesthesia.

• … in agreement with Edquist et al. (2006).

• It is dangerous to use sample size = 1.

Implications

(…what does it mean)

Sensory Sensory

Models of synaesthesia

Inducer Concurrent

Definicije sinestezije

osjet koji se obično odvija u jednom osjetnom modalitetu, odvija se i kada se podražuje drugi modalitet

izmješanost osjetila

percipiranje sensornih podataka jednoga osjetila sa drugimjedno sensorno iskustvo opisano u terminima drugog

križanje osjetnih percepcija

Sensory Sensory

Implications

Models of synaesthesia

Inducer Concurrent

Serial search task

Colour opponency

task

Sensory Sensory

- Much evidence about sensory concurrent

Inducer Concurrent

- Evidence about sensory inducer?

?

Massive evidence for the role of semantic inducers

Dixon et al. (2000): 5 + 2 =

Simner and Ward (2006) Taste of words on the tip of the tongue.

Simner: crossover of senses

Berteletti et al.: dice

All our synaesthetes reported, that the shape “O” changes its colour depending on the context, i.e., whether it is perceived as a letter or as a digit zero. Therefore, the meaning determines the colour.

A E I O U O 1 2 3 4 5

Test for context

Three homographs (same spelling, different pronunciation, different meaning): - Montage (installation or Monday)- Collagen (colleges or binding agent)- Heroin (narcotic or female hero)

The colours depended on the pronunciation/meaning.

Studija III: Semantička reprezentacija


Induktor

Dodatno iskustvo

Sensorno iskustvo

Sensorno iskustvo

Test učenjem:

Novo slovo

Crvena bojaA

Glagoljica

Od 16 ispitanika, 14 je izjavilo da je nakon kraće ili duže vježbe novo glagoljičko slovo dobilo sinestetsku boju originalnog latiničkog slova.

Obično je 5 do 10 minuta dovoljno.


Induktor

Dodatno iskustvo

Sensorno iskustvo

Sensorno iskustvo

Zaključak:

Sensorno iskustvo


Sensorno iskustvo

Klasični model:

Rezultati sugeriraju:


Induktor

Dodatno iskustvo

Sensorno iskustvo

Sensorno iskustvo

Definicije sinestezije

osjet koji se obično odvija u jednom osjetnom modalitetu, odvija se i kada se podražuje drugi modalitet

izmješanost osjetila

percipiranje sensornih podataka jednoga osjetila sa drugimjedno sensorno iskustvo opisano u terminima drugog

križanje osjetnih percepcija

SinestezijaPercepcija značenja

Osjećanje koncepata

Za ne-sinestete koncepti us abstraktni

Sinestete: koncepti postaju konkretna perceptivna iskustva

(zamenke, imena, melodije)

Sinestezija

Grčki: (syn) ujedinjenje + (aesthesis) osjeti

“Ujedinjenje osjetila”

“Osjećanje ideja”

Grčki (idea) ideja, koncept + (aesthesis) osjeti

Ideaestezija

Sensory Sensory

Model of synaesthesia

Semantic

Inducer

Concurrent

Sensory Sensory

Semantic !

Hvala na pažnji

Documents

Vidna kognicija IV