GENTS A INTELLIGENT 2: LECTUREparsons/courses/840-fall-2003/notes/le… · 1.1 Reactivity If a prog r am’ s en vironment is guar anteed to be ﬁx ed, the prog r am need ne v er

LECTURE2:INTELLIGENTAGENTS

AnIntroductiontoMultiagentSystems

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/

Lecture2AnIntroductiontoMultiagentSystems

1WhatisanAgent?

�Themainpointaboutagentsistheyareautonomous:capableofactingindependently,exhibitingcontrolovertheirinternalstate.

�Thus:anagentisacomputersystemcapableofautonomousactioninsomeenvironment.

ENVIRONMENT

SYSTEM

output input

http://www.csc.liv.ac.uk/˜mjw/pubs/imas/1


�Trivial(non-interesting)agents:

–thermostat;

–UNIXdaemon(e.g.,biff).

�Anintelligentagentisacomputersystemcapableofflexibleautonomousactioninsomeenvironment.

Byflexible,wemean:

–reactive;

–pro-active;

–social.



1.1Reactivity

�Ifaprogram’senvironmentisguaranteedtobefixed,theprogramneedneverworryaboutitsownsuccessorfailure—programjustexecutesblindly.

Exampleoffixedenvironment:compiler.

�Therealworldisnotlikethat:thingschange,informationisincomplete.Many(most?)interestingenvironmentsaredynamic.

�Softwareishardtobuildfordynamicdomains:programmusttakeintoaccountpossibilityoffailure—askitselfwhetheritisworthexecuting!

�Areactivesystemisonethatmaintainsanongoinginteractionwithitsenvironment,andrespondstochangesthatoccurinit(intimefortheresponsetobeuseful).



1.2Proactiveness

�Reactingtoanenvironmentiseasy(e.g.,stimulus�responserules).

�Butwegenerallywantagentstodothingsforus.

�Hencegoaldirectedbehaviour.

�Pro-activeness=generatingandattemptingtoachievegoals;notdrivensolelybyevents;takingtheinitiative.

�Recognisingopportunities.



1.3SocialAbility

�Therealworldisamulti-agentenvironment:wecannotgoaroundattemptingtoachievegoalswithouttakingothersintoaccount.

�Somegoalscanonlybeachievedwiththecooperationofothers.

�Similarlyformanycomputerenvironments:witnesstheINTERNET.

�Socialabilityinagentsistheabilitytointeractwithotheragents(andpossiblyhumans)viasomekindofagent-communicationlanguage,andperhapscooperatewithothers.



2OtherProperties

�Otherproperties,sometimesdiscussedinthecontextofagency:

–mobility:theabilityofanagenttomovearoundanelectronicnetwork;

–veracity:anagentwillnotknowinglycommunicatefalseinformation;

–benevolence:agentsdonothaveconflictinggoals,andthateveryagentwillthereforealwaystrytodowhatisaskedofit;

–rationality:agentwillactinordertoachieveitsgoals,andwillnotactinsuchawayastopreventitsgoalsbeingachieved—atleastinsofarasitsbeliefspermit;

–learning/adaption:agentsimproveperformanceovertime.



2.1AgentsandObjects

�Areagentsjustobjectsbyanothername?

�Object:

–encapsulatessomestate;

–communicatesviamessagepassing;

–hasmethods,correspondingtooperationsthatmaybeperformedonthisstate.



�Maindifferences:

–agentsareautonomous:agentsembodystrongernotionofautonomythanobjects,andinparticular,theydecideforthemselveswhetherornottoperformanactiononrequestfromanotheragent;

–agentsaresmart:capableofflexible(reactive,pro-active,social)behavior,andthestandardobjectmodelhasnothingtosayaboutsuchtypesofbehavior;

–agentsareactive:amulti-agentsystemisinherentlymulti-threaded,inthateachagentisassumedtohaveatleastonethreadofactivecontrol.



Objectsdoitforfree...

�agentsdoitbecausetheywantto;

�agentsdoitformoney.



2.2AgentsandExpertSystems

�Aren’tagentsjustexpertsystemsbyanothername?

�Expertsystemstypicallydisembodied‘expertise’aboutsome(abstract)domainofdiscourse(e.g.,blooddiseases).

�Example:MYCINknowsaboutblooddiseasesinhumans.

Ithasawealthofknowledgeaboutblooddiseases,intheformofrules.

AdoctorcanobtainexpertadviceaboutblooddiseasesbygivingMYCINfacts,answeringquestions,andposingqueries.



�Maindifferences:

–agentssituatedinanenvironment:MYCINisnotawareoftheworld—onlyinformationobtainedisbyaskingtheuserquestions.

–agentsact:MYCINdoesnotoperateonpatients.

�Somereal-time(typicallyprocesscontrol)expertsystemsareagents.



2.3IntelligentAgentsandAI

�Aren’tagentsjusttheAIproject?

Isn’tbuildinganagentwhatAIisallabout?

�AIaimstobuildsystemsthatcan(ultimately)understandnaturallanguage,recogniseandunderstandscenes,usecommonsense,thinkcreatively,etc—allofwhichareveryhard.

�So,don’tweneedtosolveallofAItobuildanagent...?



�Whenbuildinganagent,wesimplywantasystemthatcanchoosetherightactiontoperform,typicallyinalimiteddomain.

�WedonothavetosolvealltheproblemsofAItobuildausefulagent:

alittleintelligencegoesalongway!

�OrenEtzioni,speakingaboutthecommercialexperienceofNETBOT,Inc:

Wemadeouragentsdumberanddumberanddumber...untilfinallytheymademoney.



3Environments

�Accessiblevsinaccessible.

Anaccessibleenvironmentisoneinwhichtheagentcanobtaincomplete,accurate,up-to-dateinformationabouttheenvironment’sstate.

Mostmoderatelycomplexenvironments(including,forexample,theeverydayphysicalworldandtheInternet)areinaccessible.

Themoreaccessibleanenvironmentis,thesimpleritistobuildagentstooperateinit.



�Deterministicvsnon-deterministic.

Aswehavealreadymentioned,adeterministicenvironmentisoneinwhichanyactionhasasingleguaranteedeffect—thereisnouncertaintyaboutthestatethatwillresultfromperforminganaction.

Thephysicalworldcantoallintentsandpurposesberegardedasnon-deterministic.

Non-deterministicenvironmentspresentgreaterproblemsfortheagentdesigner.



�Episodicvsnon-episodic.

Inanepisodicenvironment,theperformanceofanagentisdependentonanumberofdiscreteepisodes,withnolinkbetweentheperformanceofanagentindifferentscenarios.

Episodicenvironmentsaresimplerfromtheagentdeveloper’sperspectivebecausetheagentcandecidewhatactiontoperformbasedonlyonthecurrentepisode—itneednotreasonabouttheinteractionsbetweenthisandfutureepisodes.



�Staticvsdynamic.

Astaticenvironmentisonethatcanbeassumedtoremainunchangedexceptbytheperformanceofactionsbytheagent.

Adynamicenvironmentisonethathasotherprocessesoperatingonit,andwhichhencechangesinwaysbeyondtheagent’scontrol.

Thephysicalworldisahighlydynamicenvironment.



�Discretevscontinuous.

Anenvironmentisdiscreteifthereareafixed,finitenumberofactionsandperceptsinit.RussellandNorviggiveachessgameasanexampleofadiscreteenvironment,andtaxidrivingasanexampleofacontinuousone.



4AgentsasIntentionalSystems

�Whenexplaininghumanactivity,itisoftenusefultomakestatementssuchasthefollowing:

Janinetookherumbrellabecauseshebelieveditwasgoingtorain.

MichaelworkedhardbecausehewantedtopossessaPhD.

�Thesestatementsmakeuseofafolkpsychology,bywhichhumanbehaviourispredictedandexplainedthroughtheattributionofattitudes,suchasbelievingandwanting(asintheaboveexamples),hoping,fearing,andsoon.

�Theattitudesemployedinsuchfolkpsychologicaldescriptionsarecalledtheintentionalnotions.



�ThephilosopherDanielDennettcoinedthetermintentionalsystemtodescribeentities‘whosebehaviourcanbepredictedbythemethodofattributingbelief,desiresandrationalacumen’.

�Dennettidentifiesdifferent‘grades’ofintentionalsystem:

‘Afirst-orderintentionalsystemhasbeliefsanddesires(etc.)butnobeliefsanddesiresaboutbeliefsanddesires....Asecond-orderintentionalsystemismoresophisticated;ithasbeliefsanddesires(andnodoubtotherintentionalstates)aboutbeliefsanddesires(andotherintentionalstates)—boththoseofothersanditsown’.

�Isitlegitimateorusefultoattributebeliefs,desires,andsoon,tocomputersystems?



�McCarthyarguedthatthereareoccasionswhentheintentionalstanceisappropriate:

‘Toascribebeliefs,freewill,intentions,consciousness,abilities,orwantstoa

machineislegitimatewhensuchanascriptionexpressesthesameinformationaboutthemachinethatitexpressesaboutaperson.Itisusefulwhenthe

ascriptionhelpsusunderstandthestructureofthemachine,itspastorfuture

behaviour,orhowtorepairorimproveit.Itisperhapsneverlogicallyrequiredevenforhumans,butexpressingreasonablybrieflywhatisactuallyknownabout

thestateofthemachineinaparticularsituationmayrequirementalqualitiesorqualitiesisomorphictothem.Theoriesofbelief,knowledgeandwantingcanbe

constructedformachinesinasimplersettingthanforhumans,andlaterappliedtohumans.Ascriptionofmentalqualitiesismoststraightforwardformachinesof

knownstructuresuchasthermostatsandcomputeroperatingsystems,butismostusefulwhenappliedtoentitieswhosestructureisincompletelyknown’.



�Whatobjectscanbedescribedbytheintentionalstance?

�Asitturnsout,moreorlessanythingcan...consideralightswitch:

‘Itisperfectlycoherenttotreatalightswitchasa(verycooperative)agentwiththecapabilityoftransmittingcurrentatwill,whoinvariablytransmitscurrentwhenitbelievesthatwewantittransmittedandnototherwise;flickingtheswitchissimplyourwayofcommunicatingourdesires’.(YoavShoham)

�Butmostadultswouldfindsuchadescriptionabsurd!

Whyisthis?



�Theanswerseemstobethatwhiletheintentionalstancedescriptionisconsistent,

...itdoesnotbuyusanything,sinceweessentiallyunderstandthemechanismsufficientlytohaveasimpler,mechanisticdescriptionofitsbehaviour.(YoavShoham)

�Putcrudely,themoreweknowaboutasystem,thelessweneedtorelyonanimistic,intentionalexplanationsofitsbehaviour.

�Butwithverycomplexsystems,amechanistic,explanationofitsbehaviourmaynotbepracticable.

�Ascomputersystemsbecomeevermorecomplex,weneedmorepowerfulabstractionsandmetaphorstoexplaintheiroperation—lowlevelexplanationsbecomeimpractical.

Theintentionalstanceissuchanabstraction.



�Theintentionalnotionsarethusabstractiontools,whichprovideuswithaconvenientandfamiliarwayofdescribing,explaining,andpredictingthebehaviourofcomplexsystems.

�Remember:mostimportantdevelopmentsincomputingarebasedonnewabstractions:

–proceduralabstraction;

–abstractdatatypes;

–objects.

Agents,andagentsasintentionalsystems,representafurther,andincreasinglypowerfulabstraction.

�Soagenttheoristsstartfromthe(strong)viewofagentsasintentionalsystems:onewhosesimplestconsistentdescriptionrequirestheintentionalstance.



�Thisintentionalstanceisanabstractiontool—aconvenientwayoftalkingaboutcomplexsystems,whichallowsustopredictandexplaintheirbehaviourwithouthavingtounderstandhowthemechanismactuallyworks.

�Now,muchofcomputerscienceisconcernedwithlookingforabstractionmechanisms(witnessproceduralabstraction,ADTs,objects,...)

Sowhynotusetheintentionalstanceasanabstractiontoolincomputing—toexplain,understand,and,crucially,programcomputersystems?

�Thisisanimportantargumentinfavourofagents.



�Other3pointsinfavourofthisidea:

CharacterisingAgents

�Itprovidesuswithafamiliar,non-technicalwayofunderstanding&explaingagents.

NestedRepresentations

�Itgivesusthepotentialtospecifysystemsthatincluderepresentationsofothersystems.

Itiswidelyacceptedthatsuchnestedrepresentationsareessentialforagentsthatmustcooperatewithotheragents.



Post-DeclarativeSystems

�Thisviewofagentsleadstoakindofpost-declarativeprogramming:

–inproceduralprogramming,wesayexactlywhatasystemshoulddo;

–indeclarativeprogramming,westatesomethingthatwewanttoachieve,givethesystemgeneralinfoabouttherelationshipsbetweenobjects,andletabuilt-incontrolmechanism(e.g.,goal-directedtheoremproving)figureoutwhattodo;

–withagents,wegiveaveryabstractspecificationofthesystem,andletthecontrolmechanismfigureoutwhattodo,knowingthatitwillactinaccordancewithsomebuilt-intheoryofagency(e.g.,thewell-knownCohen-Levesquemodelofintention).



Anaside...

�Wefindthatresearchersfromamoremainstreamcomputingdisciplinehaveadoptedasimilarsetofideas...

�Indistributedsystemstheory,logicsofknowledgeareusedinthedevelopmentofknowledgebasedprotocols.

�Therationaleisthatwhenconstructingprotocols,oneoftenencountersreasoningsuchasthefollowing:

IFprocessiknowsprocessjhasreceivedmessagem�

THENprocessishouldsendprocessjthemessagem�.

�InDStheory,knowledgeisgrounded—givenapreciseinterpretationintermsofthestatesofaprocess;returntothislater...we’llexaminethispointindetaillater.



5AbstractArchitecturesforAgents

�AssumetheenvironmentmaybeinanyofafinitesetEofdiscrete,instantaneousstates:

E��e�e��

�Agentsareassumedtohavearepertoireofpossibleactionsavailabletothem,whichtransformthestateoftheenvironment.

Ac��

�Arun,r,ofanagentinanenvironmentisasequenceofinterleavedenvironmentstatesandactions:

re�� e�

�� e�

�� e�� u�� eu



�Let:

–�bethesetofallsuchpossiblefinitesequences(overEandAc);

–�

Acbethesubsetofthesethatendwithanaction;and

–�

Ebethesubsetofthesethatendwithanenvironmentstate.



StateTransformerFunctions

�Astatetransformerfunctionrepresentsbehaviouroftheenvironment:

��

Ac

��E�

�Notethatenvironmentsare...

–historydependent.

–non-deterministic.

�If��r��,thentherearenopossiblesuccessorstatestor.Inthiscase,wesaythatthesystemhasendeditsrun.

�Formally,wesayanenvironmentEnvisatripleEnv��E�e��

where:Eisasetofenvironmentstates,e�Eistheinitialstate;and�isastatetransformerfunction.



Agents

�Agentisafunctionwhichmapsrunstoactions:

Ag�

E�Ac

Anagentmakesadecisionaboutwhatactiontoperformbasedonthehistoryofthesystemthatithaswitnessedtodate.Let��

bethesetofallagents.



Systems

�Asystemisapaircontaininganagentandanenvironment.

�Anysystemwillhaveassociatedwithitasetofpossibleruns;wedenotethesetofrunsofagentAginenvironmentEnvby

��Ag�Env�.

�(Weassume��Ag�Env�containsonlyterminatedruns.)



�Formally,asequence

�e� ��e�� e��

representsarunofanagentAginenvironmentEnv��E�e��if:

1.eistheinitialstateofEnv

2.��Ag�e�;and

3.foru��,

eu��e� �� u��where

�u�Ag��e� ��eu��



6PurelyReactiveAgents

�Someagentsdecidewhattodowithoutreferencetotheirhistory—theybasetheirdecisionmakingentirelyonthepresent,withnoreferenceatalltothepast.

�Wecallsuchagentspurelyreactive:

actionE�Ac

�Athermostatisapurelyreactiveagent.

action�e��

��

��

��

offife=temperatureOKonotherwise.



7Perception

�Nowintroduceperceptionsystem:

ENVIRONMENT

action

AGENT

see



�Theseefunctionistheagent’sabilitytoobserveitsenvironment,whereastheactionfunctionrepresentstheagent’sdecisionmakingprocess.

�Outputoftheseefunctionisapercept:

seeE�Per

whichmapsenvironmentstatestopercepts,andactionisnowafunction

actionPer

�

�A

whichmapssequencesofperceptstoactions.



8AgentswithState

�Wenowconsideragentsthatmaintainstate:

action see

nextstate

AGENT

ENVIRONMENT



�Theseagentshavesomeinternaldatastructure,whichistypicallyusedtorecordinformationabouttheenvironmentstateandhistory.LetIbethesetofallinternalstatesoftheagent.

�Theperceptionfunctionseeforastate-basedagentisunchanged:

seeE�Per

Theaction-selectionfunctionactionisnowdefinedasamapping

actionI�Ac

frominternalstatestoactions.Anadditionalfunctionnextisintroduced,whichmapsaninternalstateandpercepttoaninternalstate:

nextI�Per�I



8.1Agentcontrolloop

1.Agentstartsinsomeinitialinternalstatei.

2.Observesitsenvironmentstatee,andgeneratesaperceptsee�e�.

3.Internalstateoftheagentisthenupdatedvianextfunction,becomingnext�i�see�e��.

4.Theactionselectedbytheagentisaction�next�i�see�e��.

Thisactionisthenperformed.

5.Goto(2).



9TasksforAgents

�Webuildagentsinordertocarryouttasksforus.

�Thetaskmustbespecifiedbyus...

�Butwewanttotellagentswhattodowithouttellingthemhowtodoit.



9.1UtilitiesFunctionsoverStates

�Onepossibility:associateutilitieswithindividualstates—thetaskoftheagentisthentobringaboutstatesthatmaximiseutility.

�Ataskspecificationisafunction

uE�IR

whichassociatedarealnumberwitheveryenvironmentstate.



�Butwhatisthevalueofarun...

–minimumutilityofstateonrun?

–maximumutilityofstateonrun?

–sumofutilitiesofstatesonrun?

–average?

�Disadvantage:difficulttospecifyalongtermviewwhenassigningutilitiestoindividualstates.

(Onepossibility:adiscountforstateslateron.)



9.2UtilitiesoverRuns

�Anotherpossibility:assignsautilitynottoindividualstates,buttorunsthemselves:

u��IR

�Suchanapproachtakesaninherentlylongtermview.

�Othervariations:incorporateprobabilitiesofdifferentstatesemerging.

�Difficultieswithutility-basedapproaches:

–wheredothenumberscomefrom?

–wedon’tthinkintermsofutilities!

–hardtoformulatetasksintheseterms.



UtilityintheTileworld

�Simulatedtwodimensionalgridenvironmentonwhichthereareagents,tiles,obstacles,andholes.

�Anagentcanmoveinfourdirections,up,down,left,orright,andifitislocatednexttoatile,itcanpushit.

�Holeshavetobefilledupwithtilesbytheagent.Anagentscorespointsbyfillingholeswithtiles,withtheaimbeingtofillasmanyholesaspossible.

�TILEWORLDchangeswiththerandomappearanceanddisappearanceofholes.

�Utilityfunctiondefinedasfollows:

u�r�

��numberofholesfilledinr

numberofholesthatappearedinr



9.3ExpectedUtility&OptimalAgents

�WriteP�r�Ag�Env�todenoteprobabilitythatrunroccurswhenagentAgisplacedinenvironmentEnv.

Note:�

r��Ag�Env�P�r�Ag�Env��

�ThenoptimalagentAgoptinanenvironmentEnvistheonethatmaximizesexpectedutility:

Agopt�� Ag� ��

r��Ag�Env�

u�r�P�r�Ag�Env� �(1)



9.4BoundedOptimalAgents

�Someagentscannotbeimplementedonsomecomputers

(AfunctionAg�E�Acmayneedmorethanavailablememory

toimplement.)

�Write��mtodenotetheagentsthatcanbeimplementedonmachine(computer)m:

��m��Ag�Ag��andAgcanbeimplementedonm��

�Wecanreplaceequation(1)withthefollowing,whichdefinestheboundedoptimalagentAgopt:

Agopt�� Ag� �m�

r��Ag�Env�

u�r�P�r�Ag�Env� �(2)



9.5PredicateTaskSpecifications

�Aspecialcaseofassigningutilitiestohistoriesistoassign0(false)or1(true)toarun.

�Ifarunisassigned1,thentheagentsucceedsonthatrun,otherwiseitfails.

�Callthesepredicatetaskspecifications.

�Denotepredicatetaskspecificationby�.

Thus��.



9.6TaskEnvironments

�Ataskenvironmentisapair�Env��,whereEnvisanenvironment,and

��

isapredicateoverruns.

Let��bethesetofalltaskenvironments.

�Ataskenvironmentspecifies:

–thepropertiesofthesystemtheagentwillinhabit;

–thecriteriabywhichanagentwillbejudgedtohaveeitherfailedorsucceeded.



�Write��Ag�Env�todenotesetofallrunsoftheagentAginenvironmentEnvthatsatisfy�:

��Ag�Env��r�r��Ag�Env�and��r��

�WethensaythatanagentAgsucceedsintaskenvironment

�Env��if

��Ag�Env��Ag�Env�



TheProbabilityofSuccess

�LetP�r�Ag�Env�denoteprobabilitythatrunroccursifagentAgisplacedinenvironmentEnv.

�ThentheprobabilityP��Ag�Env�that�issatisfiedbyAginEnvwouldthensimplybe:

P��Ag�Env��

r��Ag�Env�P�r�Ag�Env�



Achievement&MaintenanceTasks

�Twomostcommontypesoftasksareachievementtasksandmaintenancetasks:

1.AchievementtasksArethoseoftheform“achievestateofaffairs�”.

2.MaintenancetasksArethoseoftheform“maintainstateofaffairs�”.



�AnachievementtaskisspecifiedbyasetGof“good”or“goal”states:G�E.

Theagentsucceedsifitisguaranteedtobringaboutatleastoneofthesestates(wedonotcarewhichone—theyareallconsideredequallygood).

�AmaintenancegoalisspecifiedbyasetBof“bad”states:B�E.

TheagentsucceedsinaparticularenvironmentifitmanagestoavoidallstatesinB—ifitneverperformsactionswhichresultinanystateinBoccurring.



10AgentSynthesis

�Agentsynthesisisautomaticprogramming:goalistohaveaprogramthatwilltakeataskenvironment,andfromthistaskenvironmentautomaticallygenerateanagentthatsucceedsinthisenvironment:

syn� ��

(Thinkof�asbeinglikenullinJAVA.

�Synthesisalgorithmis:

–soundif,wheneveritreturnsanagent,thenthisagentsucceedsinthetaskenvironmentthatispassedasinput;and

–completeifitisguaranteedtoreturnanagentwheneverthereexistsanagentthatwillsucceedinthetaskenvironmentgivenasinput.



�Synthesisalgorithmsynissoundifitsatisfiesthefollowingcondition:

syn��Env��Agimplies��Ag�Env��Ag�Env� �

andcompleteif:

�Ag��s.t.��Ag�Env��Ag�Env�impliessyn��Env��


Documents

GENTS A INTELLIGENT 2: LECTUREparsons/courses/840-fall-2003/notes/le… · 1.1 Reactivity If a prog r am’ s en vironment is guar anteed to be ﬁx ed, the prog r am need ne v er