Informatics 1 CG: Lecture 15 · Summary • Inductive reasoning is everywhere: • To perceive,...

InductivereasoningInformatics1CG:Lecture15

ChrisLucasclucas2@inf.ed.ac.uk

Knowledge

Howdoweacquireknowledge?

[Thepovertyofofstimulus]

“Howcomesitthathumanbeings,whosecontactswiththeworldarebriefandpersonalandlimited,areneverthelessabletoknowasmuchastheydoknow?”

(BertrandRussell)

Knowledge

Wheredoesknowledgecomefrom?

(1) Non-experientialsources(innate)(2) Perceptionandmemory(3) Deduction(4) Induction

Knowledge

Inductivereasoning:• Reasoningaboutcasesorprinciplesthatgobeyondcurrentdata.• Couldbewrong;entailuncertaintyorambiguity.• Havingseenonlyblackravens,wemightexpectanew,hiddenraventobeblack.• Anewravencouldbewhite!

• E.g.,Hume:“instancesofwhichwehavehadnoexperiencemustresemblethoseofwhichwehavehadexperience.”

(Hume, 1777/1975)

PerceptionJubalcalledout,"Thatnewhouseonthefarhilltop- canyouseewhatcolorthey'vepaintedit?”

AnnelookedinthedirectioninwhichJubalwaspointingandanswered,"It'swhiteonthisside."ShedidnotinquirewhyJubalhadasked,normakeanycomment.

JubalwentontoJillinnormaltones."Yousee?[…]itdoesn'tevenoccurtohertoinferthattheothersideisprobablywhitetoo…[evenifshesawit]shewouldn'tassumethatitstayed[thatcolor]...becausetheymightrepaintitassoonassheturnedherback.

(Heinlein,1961)

Perception

http://web.mit.edu/persci/people/adelson/images/checkershadow/

Perception

http://web.mit.edu/persci/people/adelson/images/checkershadow/

Memory

Serialreproductionexperiments:

“Person1:Drawthis!”

(Bartlett,1932viaXu&Griffiths,2010)

Memory

Serialreproductionexperiments:

Person1drewthis.“Person2:Drawthis!”

(Bartlett,1932viaXu&Griffiths,2010)

Memory

Serialreproductionexperiments:

Person2drewthis.“Person3:Drawthis!”

(Bartlett,1932viaXu&Griffiths,2010)

Memory

Serialreproductionexperiments:

Person3drewthis.“Person4:Drawthis!”

(Bartlett,1932viaXu&Griffiths,2010)

Memory

Serialreproductionexperiments:

(Bartlett,1932viaXu&Griffiths,2010)

Deduction

Asyllogism:

1. Allhumansaremortal.

2. Ashishuman.

3. Therefore,Ashismortal.

Deduction

Asyllogism:

1. Allhumansaremortal.

2. Ashishumanakillerandroid

3. Therefore,…?

Perfectcertaintyisrareintherealworld.

(*Hyperdyne Systemsmodel 120-A/2; http://alienanthology.wikia.com/wiki/Ash)

Induction

• Wecategorise objectswe’veneverseenbefore

• Wecanmakesenseofnewandambiguoussentences:“Ioncesawadeerridingmybicycle.”

• Wecanlearnhowtousenewtoolsandtechnology

• Wemakenewscientificdiscoveries

(https://en.wikipedia.org/wiki/List_of_linguistic_example_sentences)

Induction

Howisallthispossible?

Induction

Therationalist answer:• Themindhaslotsofinnatestructure.• Knowledgecomesfromthisstructureanditsinteractionswithexperience.

Induction

Therationalist answer:• Themindhaslotsofinnatestructure.• Knowledgecomesfromthisstructureanditsinteractionswithexperience.

Theempiricistanswer:• Thestimulusisnotasimpoverishedasonemightthink.• Wecanlearnandgeneralise withminimalinnatestructure.

Inductivebiases

Consensus:• Peoplelearnsomething fromexperience.• Weneedtohavesome startingknowledge(orassumptions)togeneralise atall.

Wecancallthis“knowledge”ourinductivebiases.

Inductivebiases

Wecanthinkofmanyquestionsincognitivescienceintermof:

(1) Whatinductivebiasesshapehumanbehaviour?

(2) Wheredoourinductivebiasescomefrom?

Inductivebiases

Inductivebiasescanbeunderstoodindifferentways:

Theycanbemadeexplicit,e.g.,• Tacitknowledgeofa“universalgrammar”• Assumptionthatcategoriesaredefinedintermsofaprototype• Assumptionthatcategorieshavehierarchicalstructure

Inductivebiases

Inductivebiasescanbeunderstoodindifferentways:

Theycanarisefromalearner’sstructureoritsenvironment:• Thearchitectureofaneuralnetwork• Thekindsofexperiencesaninfanthas

Inductivebiases

Thesearen’tmutuallyexclusive.

Aperson’sinductivebiasesmight• beaccuratelycapturedbyrulesorprobabilitydistributionsand• emergefromcomplexbiologicalphenomena.

Example:Inductivebiasesofaneuralnet

Example:Inductivebiasesofaneuralnet

(https://en.wikipedia.org/wiki/DeepDream)

Example:Inductivebiasesofaneuralnet

(https://en.wikipedia.org/wiki/DeepDream)

Example:Inductivebiasesofaneuralnet

(https://en.wikipedia.org/wiki/DeepDream)

Example:Inductivebiasesofaneuralnet

(https://en.wikipedia.org/wiki/DeepDream)

Howcanwestudyinductivebiases?

Someoptions:

1. Lookatwhatpeoplefindsurprising.2. Lookatwhatpatternspeoplefindeasy/hardtolearn.3. Predictspecifichumanjudgmentsandgeneralizations.4. Lookatstatisticalregularitiesinhumanjudgments.5. Understandwhatconstraintsbiologyimposes.

Howcanwestudyinductivebiases?

1. Lookatwhatpeoplefindsurprising.

“If<X>isinnate,infantsshouldbesurprisedby<Y>.”

Howcanwestudyinductivebiases?

2.Lookatwhatpatternspeoplefindeasy/hardtolearn.

“Ifpeoplerelyonrepresentation<X>,theyshouldcommitthefollowingkindsoferrors...”

Howcanwestudyinductivebiases?3.Predictspecifichumanjudgmentsandgeneralizations.

Howcanwestudyinductivebiases?

n = 1 n = 2 n = 3 n = 4 n = 5 n = 6 n = 7 n = 8 n = 9

Random4.Lookatstatisticalregularitiesinhumanjudgments.

Howcanwestudyinductivebiases?

5. Understandwhatconstraintsbiologyimposes

Neuroscience!Examples:• Multi-unitrecording• Modelsofcellsandcellpopulations• Geneticmanipulation(e.g.,knockoutsandoptogenetics)

(Forreadingsonthis,seehttp://homepages.inf.ed.ac.uk/pseries/ccn16.htm)

http://web.stanford.edu/group/dlab/optogenetics/

Summary

• Inductivereasoningiseverywhere:• Toperceive,understand,andactontheworldaroundus,wemustgeneralise,usinginformationthatisnoisy,incomplete,andambiguous.

• Generalisation isimpossiblewithoutinductivebiases.• Inductivebiasesincludeacquiredknowledgeandbiologicalconstraints.• Manybigquestionsincognitivesciencecanbeframedintermsof:• Whatareourinductivebiases?• Wheredotheycomefrom?