CS 416Artificial Intelligence

Lecture 2Lecture 2

IntroductionIntroduction

Lecture 2Lecture 2

IntroductionIntroduction

CS at UVa

$11M in research grants each year$11M in research grants each year

• Top 5% of research is funded by NSFTop 5% of research is funded by NSF

• Faculty trips to NSF set national funding prioritiesFaculty trips to NSF set national funding priorities

• Free MSFT Visual Studio for all studentsFree MSFT Visual Studio for all students

75% faculty growth in past six years75% faculty growth in past six years

Undergrad research awards from CRAUndergrad research awards from CRA

Highest starting salary (in SEAS) for ugrads Highest starting salary (in SEAS) for ugrads

$11M in research grants each year$11M in research grants each year

• Top 5% of research is funded by NSFTop 5% of research is funded by NSF

• Faculty trips to NSF set national funding prioritiesFaculty trips to NSF set national funding priorities

• Free MSFT Visual Studio for all studentsFree MSFT Visual Studio for all students

75% faculty growth in past six years75% faculty growth in past six years

Undergrad research awards from CRAUndergrad research awards from CRA

Highest starting salary (in SEAS) for ugrads Highest starting salary (in SEAS) for ugrads

Textbook

This is a great bookThis is a great book• 22ndnd edition released three years ago edition released three years ago

• Most widely used in U.S. universitiesMost widely used in U.S. universities

• It’s so good….It’s so good….

– I’m going to make you read it!I’m going to make you read it!

HomeworkHomework• Read chapters 1 and 2Read chapters 1 and 2

This is a great bookThis is a great book• 22ndnd edition released three years ago edition released three years ago

• Most widely used in U.S. universitiesMost widely used in U.S. universities

• It’s so good….It’s so good….

– I’m going to make you read it!I’m going to make you read it!

HomeworkHomework• Read chapters 1 and 2Read chapters 1 and 2

Survey Results

• LanguagesLanguages

– Supermajority prefers C++Supermajority prefers C++

– Three people indicated they’ll need C++ helpThree people indicated they’ll need C++ help

– LISP?LISP?

• MathMath

– Many w/o statMany w/o stat

– 7 w/o diffyq7 w/o diffyq

– 14 w/o linear algebra14 w/o linear algebra

• LanguagesLanguages

– Supermajority prefers C++Supermajority prefers C++

– Three people indicated they’ll need C++ helpThree people indicated they’ll need C++ help

– LISP?LISP?

• MathMath

– Many w/o statMany w/o stat

– 7 w/o diffyq7 w/o diffyq

– 14 w/o linear algebra14 w/o linear algebra

• 5 people w/o GUI experience5 people w/o GUI experience

• 4 people w/o MSFT Windows4 people w/o MSFT Windows

• 14 people don’t play so many video games14 people don’t play so many video games

• Where have you done the most programming?Where have you done the most programming?

– 216 – 17216 – 17

– Graphics – 15Graphics – 15

– 201/202 – 6201/202 – 6

– OS – 2OS – 2

• 5 people w/o GUI experience5 people w/o GUI experience

• 4 people w/o MSFT Windows4 people w/o MSFT Windows

• 14 people don’t play so many video games14 people don’t play so many video games

• Where have you done the most programming?Where have you done the most programming?

– 216 – 17216 – 17

– Graphics – 15Graphics – 15

– 201/202 – 6201/202 – 6

– OS – 2OS – 2

• AI appsAI apps

– Chess, google, spam filter, finance, chatterbot, games, Chess, google, spam filter, finance, chatterbot, games, vacuumvacuum

12% of CPU for AI tasks in games?12% of CPU for AI tasks in games?

More about magic tricks than AI?More about magic tricks than AI?

• AI appsAI apps

– Chess, google, spam filter, finance, chatterbot, games, Chess, google, spam filter, finance, chatterbot, games, vacuumvacuum

12% of CPU for AI tasks in games?12% of CPU for AI tasks in games?

More about magic tricks than AI?More about magic tricks than AI?

iRoomba - Rodney Brooks’ (MIT) company

Languages• Is AI special in its PL needs?Is AI special in its PL needs?

– AI research used to be more symbolicAI research used to be more symbolic

A language had to make it easy to create symbols and to A language had to make it easy to create symbols and to manipulate themmanipulate them

Some symbols would operate on other symbolsSome symbols would operate on other symbols

LISP supported “programs as data” and dynamic typingLISP supported “programs as data” and dynamic typing

– Modern AI is more quantitativeModern AI is more quantitative

No language has emerged with an advantageNo language has emerged with an advantage

• Our language choice cannot distract from learning AIOur language choice cannot distract from learning AI

• Is AI special in its PL needs?Is AI special in its PL needs?

– AI research used to be more symbolicAI research used to be more symbolic

A language had to make it easy to create symbols and to A language had to make it easy to create symbols and to manipulate themmanipulate them

Some symbols would operate on other symbolsSome symbols would operate on other symbols

LISP supported “programs as data” and dynamic typingLISP supported “programs as data” and dynamic typing

– Modern AI is more quantitativeModern AI is more quantitative

No language has emerged with an advantageNo language has emerged with an advantage

• Our language choice cannot distract from learning AIOur language choice cannot distract from learning AI

Languages

• C++ - Common industry languageC++ - Common industry language

• C – gets a little closer to real-time OSC – gets a little closer to real-time OS

• Perl – the duct tape of the Internet – “makes the easy things easy and Perl – the duct tape of the Internet – “makes the easy things easy and the hard things impossible” – “there’s more than one way to do it”the hard things impossible” – “there’s more than one way to do it”

• Python – “there’s only one way to do it”Python – “there’s only one way to do it”

• Scheme – easy to learn but difficult to extendScheme – easy to learn but difficult to extend

• Common Lisp – “the programmable programming language” – nontrivial Common Lisp – “the programmable programming language” – nontrivial to learn but a decidedly different experience from programming in to learn but a decidedly different experience from programming in imperative languagesimperative languages

• C++ - Common industry languageC++ - Common industry language

• C – gets a little closer to real-time OSC – gets a little closer to real-time OS

• Perl – the duct tape of the Internet – “makes the easy things easy and Perl – the duct tape of the Internet – “makes the easy things easy and the hard things impossible” – “there’s more than one way to do it”the hard things impossible” – “there’s more than one way to do it”

• Python – “there’s only one way to do it”Python – “there’s only one way to do it”

• Scheme – easy to learn but difficult to extendScheme – easy to learn but difficult to extend

• Common Lisp – “the programmable programming language” – nontrivial Common Lisp – “the programmable programming language” – nontrivial to learn but a decidedly different experience from programming in to learn but a decidedly different experience from programming in imperative languagesimperative languages

What is expected of you

You’ll have to do mathYou’ll have to do math

• Neural network update functionNeural network update function

• Multidimensional function Multidimensional function minimizationminimization

• Probability – Bayes’ RuleProbability – Bayes’ Rule

• We will teach necessary parts ofWe will teach necessary parts ofstatistics and linear algebrastatistics and linear algebra

You’ll have to do mathYou’ll have to do math

• Neural network update functionNeural network update function

• Multidimensional function Multidimensional function minimizationminimization

• Probability – Bayes’ RuleProbability – Bayes’ Rule

• We will teach necessary parts ofWe will teach necessary parts ofstatistics and linear algebrastatistics and linear algebra

Tcx ji

cx

ji w

Pw

,

,

2

)(

)()|()|(

XP

YPYXPXYP

Calculus expected.Probability and Linear

Algebra beneficial.

What is expected of you

You have to programYou have to program

• The programming assignments are non-trivialThe programming assignments are non-trivial

– C++C++

– Requires integration with existing code librariesRequires integration with existing code libraries

– Input/output handling (images, for example)Input/output handling (images, for example)

– We do not teach programming in this courseWe do not teach programming in this course

You have to programYou have to program

• The programming assignments are non-trivialThe programming assignments are non-trivial

– C++C++

– Requires integration with existing code librariesRequires integration with existing code libraries

– Input/output handling (images, for example)Input/output handling (images, for example)

– We do not teach programming in this courseWe do not teach programming in this course

CS 216 expected.Additional programming

experience beneficial.

AI Systems

• ThermostatThermostat

• Tic-Tac-ToeTic-Tac-Toe

• Your carYour car

• ChessChess

• GoogleGoogle

• BabblefishBabblefish

• ThermostatThermostat

• Tic-Tac-ToeTic-Tac-Toe

• Your carYour car

• ChessChess

• GoogleGoogle

• BabblefishBabblefish• This thingThis thing

– AsimoAsimo

• This thingThis thing

– AsimoAsimo

Examples

• Chess: Deep Junior (IBM) tied Kasparov in 2003 matchChess: Deep Junior (IBM) tied Kasparov in 2003 match• Chess: Deep Junior (IBM) tied Kasparov in 2003 matchChess: Deep Junior (IBM) tied Kasparov in 2003 match

ATR’s DB Android

Honda’s Asimo

Ritsumeikan University

RHex Hexapod

AI Techniques

• Rule-basedRule-based

• Fuzzy LogicFuzzy Logic

• Neural NetworksNeural Networks

• Genetic AlgorithmsGenetic Algorithms

• Exhaustive searchExhaustive search

• Expert SystemsExpert Systems

• LogicLogic

• Rule-basedRule-based

• Fuzzy LogicFuzzy Logic

• Neural NetworksNeural Networks

• Genetic AlgorithmsGenetic Algorithms

• Exhaustive searchExhaustive search

• Expert SystemsExpert Systems

• LogicLogic

How to Categorize These Systems

Systems that think like humansSystems that think like humans

Systems that act like humansSystems that act like humans

Systems that think rationallySystems that think rationally

Systems that act rationallySystems that act rationally

Systems that think like humansSystems that think like humans

Systems that act like humansSystems that act like humans

Systems that think rationallySystems that think rationally

Systems that act rationallySystems that act rationally

Systems that think/act like humans

It’s hard to study things you can’t observe…It’s hard to study things you can’t observe…

• How can I know how you think?How can I know how you think?

– Observation is difficult (changing with fMRI). For the most part, you Observation is difficult (changing with fMRI). For the most part, you are a “black box”are a “black box”

– Cognitive ScienceCognitive Science

• How can I know how you act?How can I know how you act?

– Observation is possible, but hard to control all aspects of Observation is possible, but hard to control all aspects of experimental conditions.experimental conditions.

– Turing TestTuring Test

It’s hard to study things you can’t observe…It’s hard to study things you can’t observe…

• How can I know how you think?How can I know how you think?

– Observation is difficult (changing with fMRI). For the most part, you Observation is difficult (changing with fMRI). For the most part, you are a “black box”are a “black box”

– Cognitive ScienceCognitive Science

• How can I know how you act?How can I know how you act?

– Observation is possible, but hard to control all aspects of Observation is possible, but hard to control all aspects of experimental conditions.experimental conditions.

– Turing TestTuring Test

Alan Turing – “Building a Brain”

World War II motivated computer advancesWorld War II motivated computer advances

• Code breaking (1943, Colossus) – Used to decipher Code breaking (1943, Colossus) – Used to decipher telegrams encrypted using Germany’s encryption machinetelegrams encrypted using Germany’s encryption machine

• Electronic Numerical Integrator and Computer (ENIAC, 1946)Electronic Numerical Integrator and Computer (ENIAC, 1946)

Turing greatly involved with British efforts to build Turing greatly involved with British efforts to build computers and crack codes (Bletchley Park)computers and crack codes (Bletchley Park)

• Arrested for being a homosexual in 1952 and denied security clearanceArrested for being a homosexual in 1952 and denied security clearance

• Committed suicide in 1954Committed suicide in 1954

World War II motivated computer advancesWorld War II motivated computer advances

• Code breaking (1943, Colossus) – Used to decipher Code breaking (1943, Colossus) – Used to decipher telegrams encrypted using Germany’s encryption machinetelegrams encrypted using Germany’s encryption machine

• Electronic Numerical Integrator and Computer (ENIAC, 1946)Electronic Numerical Integrator and Computer (ENIAC, 1946)

Turing greatly involved with British efforts to build Turing greatly involved with British efforts to build computers and crack codes (Bletchley Park)computers and crack codes (Bletchley Park)

• Arrested for being a homosexual in 1952 and denied security clearanceArrested for being a homosexual in 1952 and denied security clearance

• Committed suicide in 1954Committed suicide in 1954

Systems that think/act rationally

Rely on logic itself rather than human to Rely on logic itself rather than human to measure correctnessmeasure correctness

• Thinking rationally (logically)Thinking rationally (logically)

– Socrates is a human; All humans are mortal; Socrates is mortalSocrates is a human; All humans are mortal; Socrates is mortal

– Logic formulas for synthesizing outcomesLogic formulas for synthesizing outcomes

• Acting rationally (logically)Acting rationally (logically)

– Even if method is illogical, the observed behavior must be Even if method is illogical, the observed behavior must be rationalrational

Rely on logic itself rather than human to Rely on logic itself rather than human to measure correctnessmeasure correctness

• Thinking rationally (logically)Thinking rationally (logically)

– Socrates is a human; All humans are mortal; Socrates is mortalSocrates is a human; All humans are mortal; Socrates is mortal

– Logic formulas for synthesizing outcomesLogic formulas for synthesizing outcomes

• Acting rationally (logically)Acting rationally (logically)

– Even if method is illogical, the observed behavior must be Even if method is illogical, the observed behavior must be rationalrational

Perspective of this Course

We will investigate the general principles of We will investigate the general principles of rational agentsrational agents

• Not restricted to human actions and human environmentsNot restricted to human actions and human environments

• Not restricted to human thoughtNot restricted to human thought

• Not confined to only using laws of logicNot confined to only using laws of logic

• Anything goes so long as it produces rational behaviorAnything goes so long as it produces rational behavior

We will investigate the general principles of We will investigate the general principles of rational agentsrational agents

• Not restricted to human actions and human environmentsNot restricted to human actions and human environments

• Not restricted to human thoughtNot restricted to human thought

• Not confined to only using laws of logicNot confined to only using laws of logic

• Anything goes so long as it produces rational behaviorAnything goes so long as it produces rational behavior

What is AI?

The use of computers to solve problems that The use of computers to solve problems that previously could only be solved by applying human previously could only be solved by applying human intelligence…. thus something can fit this definition intelligence…. thus something can fit this definition today, but, once we see how the program works and today, but, once we see how the program works and understand the problem, we will not think of it as AI understand the problem, we will not think of it as AI anymoreanymore (David Parnas) (David Parnas)

The use of computers to solve problems that The use of computers to solve problems that previously could only be solved by applying human previously could only be solved by applying human intelligence…. thus something can fit this definition intelligence…. thus something can fit this definition today, but, once we see how the program works and today, but, once we see how the program works and understand the problem, we will not think of it as AI understand the problem, we will not think of it as AI anymoreanymore (David Parnas) (David Parnas)

Foundations - Philosophy

• Aristotle (384 B.C.E.) – Author of logical syllogismsAristotle (384 B.C.E.) – Author of logical syllogisms

• da Vinci (1452) – designed, but didn’t build, first mechanical da Vinci (1452) – designed, but didn’t build, first mechanical calculatorcalculator

• Descartes (1596) – can human free will be captured by a Descartes (1596) – can human free will be captured by a machine? Is animal behavior more mechanistic?machine? Is animal behavior more mechanistic?

• Necessary connection between logic and action is Necessary connection between logic and action is discovereddiscovered

• Aristotle (384 B.C.E.) – Author of logical syllogismsAristotle (384 B.C.E.) – Author of logical syllogisms

• da Vinci (1452) – designed, but didn’t build, first mechanical da Vinci (1452) – designed, but didn’t build, first mechanical calculatorcalculator

• Descartes (1596) – can human free will be captured by a Descartes (1596) – can human free will be captured by a machine? Is animal behavior more mechanistic?machine? Is animal behavior more mechanistic?

• Necessary connection between logic and action is Necessary connection between logic and action is discovereddiscovered

Foundations - Mathematics

• Leveraging uncertainty (Cardano 1501)Leveraging uncertainty (Cardano 1501)

• Boolean logic (Boole, 1847)Boolean logic (Boole, 1847)

• Analysis of limits to what can be computedAnalysis of limits to what can be computed

– Intractability (1965) – time required to solve problem Intractability (1965) – time required to solve problem scales exponentially with the size of problem instancescales exponentially with the size of problem instance

– NP-complete (1971) – Formal classification of problems as NP-complete (1971) – Formal classification of problems as intractableintractable

• Leveraging uncertainty (Cardano 1501)Leveraging uncertainty (Cardano 1501)

• Boolean logic (Boole, 1847)Boolean logic (Boole, 1847)

• Analysis of limits to what can be computedAnalysis of limits to what can be computed

– Intractability (1965) – time required to solve problem Intractability (1965) – time required to solve problem scales exponentially with the size of problem instancescales exponentially with the size of problem instance

– NP-complete (1971) – Formal classification of problems as NP-complete (1971) – Formal classification of problems as intractableintractable

Foundations - Economics

• Game Theory – study of rational behavior in small gamesGame Theory – study of rational behavior in small games

• Operations Research – study of rational behavior in Operations Research – study of rational behavior in complex systemscomplex systems

• Herbert Simon (1916 – 2001) – AI researcher who received Herbert Simon (1916 – 2001) – AI researcher who received Nobel Prize in Economics for showing people accomplish Nobel Prize in Economics for showing people accomplish satisficingsatisficing solutions, those that are good enough solutions, those that are good enough

• Game Theory – study of rational behavior in small gamesGame Theory – study of rational behavior in small games

• Operations Research – study of rational behavior in Operations Research – study of rational behavior in complex systemscomplex systems

• Herbert Simon (1916 – 2001) – AI researcher who received Herbert Simon (1916 – 2001) – AI researcher who received Nobel Prize in Economics for showing people accomplish Nobel Prize in Economics for showing people accomplish satisficingsatisficing solutions, those that are good enough solutions, those that are good enough

Foundations - NeuroscienceHow do brains work?How do brains work?

• Early studies (1824) relied on injured and abnormal people to understand what Early studies (1824) relied on injured and abnormal people to understand what parts of brain doparts of brain do

• More recent studies use accurate sensors to correlate brain activity to human More recent studies use accurate sensors to correlate brain activity to human thoughtthought

– By monitoring individual neurons, monkeys can now control a computer By monitoring individual neurons, monkeys can now control a computer mouse using thought alonemouse using thought alone

– Melody Moore at GaState – “locked-in syndrome”Melody Moore at GaState – “locked-in syndrome”

• (Gordon) Moore’s law states computers will have as many gates as humans (Gordon) Moore’s law states computers will have as many gates as humans have neurons in 2020have neurons in 2020

• How close are we to having a mechanical brain?How close are we to having a mechanical brain?

– Parallel computation, remapping, interconnections, binary vs. gradient…Parallel computation, remapping, interconnections, binary vs. gradient…

How do brains work?How do brains work?

• Early studies (1824) relied on injured and abnormal people to understand what Early studies (1824) relied on injured and abnormal people to understand what parts of brain doparts of brain do

• More recent studies use accurate sensors to correlate brain activity to human More recent studies use accurate sensors to correlate brain activity to human thoughtthought

– By monitoring individual neurons, monkeys can now control a computer By monitoring individual neurons, monkeys can now control a computer mouse using thought alonemouse using thought alone

– Melody Moore at GaState – “locked-in syndrome”Melody Moore at GaState – “locked-in syndrome”

• (Gordon) Moore’s law states computers will have as many gates as humans (Gordon) Moore’s law states computers will have as many gates as humans have neurons in 2020have neurons in 2020

• How close are we to having a mechanical brain?How close are we to having a mechanical brain?

– Parallel computation, remapping, interconnections, binary vs. gradient…Parallel computation, remapping, interconnections, binary vs. gradient…

Foundations - Psychology

• Helmholtz and Wundt (1821) – started to make psychology a Helmholtz and Wundt (1821) – started to make psychology a science by carefully controlling experimentsscience by carefully controlling experiments

• The brain processes information (1842)The brain processes information (1842)

– Sense Sense Think Think Act Act

– Cognitive science started at a MIT workshop in 1956 with Cognitive science started at a MIT workshop in 1956 with the publication of three very influential papersthe publication of three very influential papers

• Helmholtz and Wundt (1821) – started to make psychology a Helmholtz and Wundt (1821) – started to make psychology a science by carefully controlling experimentsscience by carefully controlling experiments

• The brain processes information (1842)The brain processes information (1842)

– Sense Sense Think Think Act Act

– Cognitive science started at a MIT workshop in 1956 with Cognitive science started at a MIT workshop in 1956 with the publication of three very influential papersthe publication of three very influential papers

Foundations – Control Theory

• Machines can modify their behavior in response to the Machines can modify their behavior in response to the environment (sense / action loop)environment (sense / action loop)

– Water-flow regulator (250 B.C.E), steam engine governor, Water-flow regulator (250 B.C.E), steam engine governor, thermostatthermostat

• The theory of stable feedback systems (1894)The theory of stable feedback systems (1894)

– Build systems that transition from initialBuild systems that transition from initialstate to goal state with minimum energystate to goal state with minimum energy

– In 1950, control theory could only describeIn 1950, control theory could only describelinear systems and AI largely rose as alinear systems and AI largely rose as aresponse to this shortcomingresponse to this shortcoming

• Machines can modify their behavior in response to the Machines can modify their behavior in response to the environment (sense / action loop)environment (sense / action loop)

– Water-flow regulator (250 B.C.E), steam engine governor, Water-flow regulator (250 B.C.E), steam engine governor, thermostatthermostat

• The theory of stable feedback systems (1894)The theory of stable feedback systems (1894)

– Build systems that transition from initialBuild systems that transition from initialstate to goal state with minimum energystate to goal state with minimum energy

– In 1950, control theory could only describeIn 1950, control theory could only describelinear systems and AI largely rose as alinear systems and AI largely rose as aresponse to this shortcomingresponse to this shortcoming

Foundations - Linguistics

Speech demonstrates so much of human Speech demonstrates so much of human intelligenceintelligence

• Analysis of human language reveals thought taking place in Analysis of human language reveals thought taking place in ways not understood in other settingsways not understood in other settings

– Children can create sentences they have never heard Children can create sentences they have never heard beforebefore

– Language and thought are believed to be tightly Language and thought are believed to be tightly intertwinedintertwined

Speech demonstrates so much of human Speech demonstrates so much of human intelligenceintelligence

• Analysis of human language reveals thought taking place in Analysis of human language reveals thought taking place in ways not understood in other settingsways not understood in other settings

– Children can create sentences they have never heard Children can create sentences they have never heard beforebefore

– Language and thought are believed to be tightly Language and thought are believed to be tightly intertwinedintertwined