Connectionism and LOTH

Connectionism

&

Language of Thought

How is connectionism an alternative to LOTH?

• LOT usually represented as implemented by “classical AI.” (Also known as GOFAI: “good, old-fashioned AI”.)

• Semantic symbols and syntactic rules are easy to represent in classic AI architecture.

• Connectionism does not require symbols, but representations can be symbolic.

Types of Connectionist Representations

1) Local representation.

Meows

Fur

Pointed ears

Whiskers

Output: “it’s a cat”

This node is a local representation of “cat”.

Characterizing local representations:

• Individual nodes are symbols, and can be components of a language of thought.

• Not typical of connectionist networks.

• One neuron per symbol does not seem biologically plausible. Cell assemblies havethus been proposed for neuro-symbols:http://publik.tuwien.ac.at/files/PubDat_166316.pdf

2) Distributed representations.

E.g.:

See also:

David L. Anderson. Computer Types: Classical vs. Non-classicalhttp://www.mind.ilstu.edu/curriculum/nature_of_computers/computer_types.php

(cf. SmartKitchen: http://smartkitchen.ict.tuwien.ac.at/project/project.html)

Cat

Tiger

Leopard

Lion

Characterizing distributed representations:

• Connectionist networks are typically distributed representations.

• Distributed representations are not necessarily symbolic.

• Distributed representations are more robust to damage than local representations.

3) No representation.

More controversially, connectionist networks might have no representational properties.

Note:

• Output of connectionist network may be recognition of a concept, e.g. Cat, Tiger, Man, etc. but…

• Output of connectionist network may also be action, e.g. moving through space, reading aloud

• Rather than representing content, networks can just act.

Comparison What goes on in your mind when you

decide to drink a glass of water that is in front of you?

LOTH: the action is the conclusion of a practical syllogism conducted through symbol manipulation

Connectionism: the action is output of a neural net responding to a certain set of inputs

LOTH approach:

I am thirsty. There is a cup of water in front of me.I believe that drinking the water will relieve my thirst.(There is no reason not to drink the water)

Conclusion: I drink the water.

The conclusion is reached after manipulating the semantic symbols representing beliefs and desires in accordance with syntactic laws.

Beliefs and desires give rise to action.

Connectionist approach:

Inputs from body Inputs from environment

Output: I drink the water.

There are no symbols involved.

Connectionism makes eliminativism possible.

Note: in the connectionist/eliminativist approach, the mind concocts the belief-desire explanation, “I drank the water because I was thirsty” to explain its behavior.

But the desire (thirst) and beliefs (“the water is in front of me”, “the water is safe to drink”, “the water will relieve my thirst”) are not literally part of the process whereby the mind decides to drink.

In other words, the mind only uses symbolic representation when translating/explaining its thoughts in language (talking to oneself or talking to others).

But how can “thirst” not play a role in deciding to drink? Isn’t it part of the input from the body?

“Thirst” is a feeling. What plays the functional role of “thirst” may be a mechanism to detect that the body is low on water, or is somewhat overheated, but this may not be recognized by you as a desire, until you try to explain your own behavior.

Note: imagine reaching unconsciously for a glass of water, and when someone asks, “why are you drinking that?”, you say, “I guess I was thirsty.”

The explanation could be rather different than the cause (cf. Freud’s concept of rationalization).

Advantages of Connectionism1) Biological plausibility

Connectionist networks are deliberately analogous to neural processes in the brain

Units ~ neuronsConnections ~ synapsesActivations ~ neural signals

Neuron Connectionist unit

2) Fast processing via parallelism• “100 Step” argument.

Neurons change state very slowly compared with computer computations. Neurons can only process 100 steps a second (whereas computers can process a million). But the brain can solve many complex problems in less than 1 second, e.g. face recognition for these, it can use maximally 100 steps.

• Conventional computer programs do mostly serial processing and usually require considerably more than 100 processing steps for problems where brains need less than a second.So, such computers cannot provide a good model of cognition.

• Connectionist computations are done by parallel processing, thus much more can be achieved in 100 steps.

Cf.: http://www.ucs.louisiana.edu/~isb9112/dept/phil341/myths/myths.html

3) Performance of connectionist networks resembles performance of human brains

Connectionist networks are good at:

• Pattern recognition: networks can learn through examples

• Content-addressable memory: items can be retrieved based on their meanings or properties

• Generalizations: networks can generalize connections between characteristics or properties

Connectionist networks exhibit:

Graceful degradation

When a connectionist network has some incorrect input -- “noisy input” -- or is itself partially damaged, it still performs, more poorly, but doesn’t completely break down.

4) Connectionism provides a naturalistic mechanism for creating concepts.

No need to posit inborn concepts.

Concepts can precede language without being inborn.

Fodor once claimed that mentalese was“the only game in town”.

Connectionism is a new game!

Criticisms of connectionismThe advantages of connectionism revisited:

1) Biological plausibility

2) 100-steps argument

3) Pattern recognition and concept formation: yes, but can be slow

1) Biological plausibility

Networks aren’t really like neurons.

• No reverse connections (necessary for backward propagation) in the brain.

• Neurons only fire or not: they cannot be both inhibitory and excitatory.

• Connectionist units are too fast, neurons are quite slow.

Biological plausibility (cont.)

• There are many different types of neurons in the brain, but connectionist units are meant to represent all neurons.

• In addition, role of neurotransmitters and hormones in thinking is ignored in connectionist models.

Note: most people admit that connectionist networks are still more biologically plausible than classical AI architectures.

Different types of neurons

2) The 100 step argument

Problem: what is a step?

Is, recognizing a color one step? Or does it break down into numerous steps?

The 100 step argument only works if each unit of a connectionist network corresponds to one neuron.

If one unit corresponds to several neurons working together, the 100 step constraint may be greatly exceeded.

Also, the 100 step argument assumes only connectionist architectures are parallel processors, while all non- connectionist architectures are serial. But it is possible to build parallel non-connectionist architectures. Cf. review:http://www.icsr.agh.edu.pl/publications/html/ppam97prof/ppam97prof.html

3) Network learning can be slow

Many connectionist networks need a large amount of explicit feedback to learn. Others, e.g. self-organizing maps, use unsupervised learning :http://www.willamette.edu/~gorr/classes/cs449/Unsupervised/SOM.html

The brain often seems to learn a new concept or pattern in one shot.

One-shot learning is especially easy when information is gathered through language.

Example: think of teaching an intelligent chimp vs. a five-year-old child, to push the red button for food.

Another weakness of Connectionism

Systematicity and productivity: very difficult (impossible?) to implement in connectionist architecture.

Connectionist responses:

1) Deny systematicity and productivity of the mind:

Is human thinking really systematic/productive?Do animals think systematically/productively?

2) Maintain the ability of connectionist nets to generate systematicity and productivity

The Relationship between Connectionism and LOTH

Three possibilities:

1) Connectionism implements LOTH2) Connectionism replaces LOTH3) Hybrid theory. Some mental processes

are connectionist, others are conducted through LOT.

1) Connectionism implements LOT

Connectionist nets can be regarded as a lower-level implementation of LOT.

Neural nets can represent semantic symbols which are then manipulated in accordance with language-like laws (also implemented by neural nets).

Criticism: if connectionist nets only implement LOT, many of the advantages of connectionism are lost.

2) Connectionism replaces LOT

Consequence: all the advantages (e.g. systematicity and productivity) of LOT are lost.

Can we do without them?

3) Hybrid theory

Some mental processes are connectionist, others are conducted through LOT.

E.g.:

Perception and motor control handled by connectionist nets.

Reasoning and language handled by LOT, and implemented by connectionist nets.

Peripheral Process Central Process

Mental Process subClassOf

Perception Motor Control Reasoning Language

LOT

Connectionist Net implement

handle

handle

Class-level logic of hybrid theory presented as a graph

Z

Y

X implement

handle

realize

Rule composing the implement and handle relations

Peripheral Process Central Process

Mental Process subClassOf

Perception Motor Control Reasoning Language

LOT

Connectionist Net implement

handle

handle

realize

Class-level logic extended by inferred relation

Connectionism and ModularityConnectionist networks can do simple, small tasks.

In more complicated tasks, they are overwhelmed by the complexity (because the connections increase exponentially).

Mind must be organized into simple units, connected up in an efficient way.

“Connectoplasm”: the mind an unorganized mess of connections. Not a viable idea.

Mental modules: some connections preset, others learned.A way to contain the complexity (maybe even recursively: modules of modules).

Readings for next weekFocus:

Thomas Nagel (1974), “What is it like to be a bat?”, The Philosophical Review, LXXXIII, 4 (October 1974), 435-50http://www.clarku.edu/students/philosophyclub/docs/nagel.pdf

Block (2002), “Some Concepts of Consciousness”, in David Chalmers (Ed.). Philosophy of Mind: Classical and Contemporary Readings Oxford University Press http://www.nyu.edu/gsas/dept/philo/faculty/block/papers/Abridged%20BBS.htm

Extra:

Gallup, Jr., Povinelli (1998). Can Animals Empathize? Yes. Scientific American - Exploring Intelligence (a debate).http://www.sciamdigital.com/index.cfm?fa=Products.ViewIssuePreview&ARTICLEID_CHAR=9123A7A5-59B3-4355-8946-C0E31A72A09