TDDD10AIProgrammingPuttingItAllTogether

CyrilleBerger

2/65

Lectures

1AIProgramming:Introduction2Introductionto3AgentsandAgents4Multi-Agentand5Multi-AgentDecision6CooperationAndCoordination7CooperationAndCoordination8MachineLearning9AutomatedPlanning10PuttingItAllTogether

3/65

Lecturegoals

Learnhowallthealgorithmsandconceptsfittogether

4/65

LecturesSummary

5/65

Lecturecontent

AgentArchitectureCommunicationRelayTrafficsurveillanceSensingStream-BasedReasoning

DetectingandTrackingcars

Multi-agentsensing

RescueoperationHumanDetectionandPlanningDelegation

BuildingScanningMasterthesis

AgentArchitecture

7

ArchitectureTheory

8

UAVArchitecture

CommunicationRelay

10

CommunicationRelay

Communicationisakeycomponentofmulti-agentsystems

Communicationcanbe

CommunicationnetworkscanbedisabledordestroyedForinstance,during9/11,cellularphoneswerenotworking

11

CommunicationRelay:multiplepath

12

CommunicationRelay:visibility

13

CommunicationRelayThegoalistofindachainofUAVthatminimizethenumberofThisisacomplicatedoptimizationGetworseifyouhavemultipleUseVoronoigraphandTreebased

Thisworkin2D,butwhatabout

14

Networkbuildingwithdelegation

NetworkbuildingmissionSequence

DeploybasestationSequence

Flyto Grab Flyto Grab

DeployrelaystationsConcurrent

Deployrelaystation1Sequence

...

15

Howtogetfullcoverage?

Findinganoptimalcoveragein3Disunreasonable

Instead,whenanagentisinanareawithlowcommunication,itshouldsendarequestforgettingarelaystationinstalled

Trafficsurveillance

17

TrafficsurveillanceScenario(1/2)

18

TrafficsurveillanceScenario(2/2)

Continuouslygatherinformationfrommanydifferentsources.

Selecttherelevantinformationforthecurrenttask.

Derivehigher-levelknowledgeabouttheenvironmentandtheUAV.Suchasdetectingmisbehavingdrivers

Correctlyinterpretwhatisgoingon.

CoordinationofUAVswithother

CoordinationofUAVswithotherhumanpoliceforces(calledmixinitiative)

Sensing Stream-BasedReasoning

21

TheSense-ReasoningGap

22

Stream-BasedReasoning

Autonomoussystemsproduceandprocesssequencesofvaluesincrementallycreatedatrun-time.

Thesesequencesarenaturaltomodelasstreams.Stream-basedreasoningisincrementalreasoningoverstreams.Stream-basedreasoningcapturesthecontinuousreasoningwithminimallatencynecessarytoreacttorapidchangesintheenvironment.

23

IncrementalEvaluationofTemporalLogicalFormulas

Thesemanticsoftheseformulasisdefinedoverinfinitestatesequences.Progressionisonetechniquetocheckwhetherthecurrentprefixissufficienttodeterminethetruthvalueofaformula.

24

StateStreams

DetectingandTrackingcars

26

TrafficMonitoring

27

Anchoring(1/3)

Theobjectiveoftheanchoringprocessistoconnectsymbolstosensordataoriginatinginthephysicalworldsothatthesymbolsrepresenttheobjectsintheworld

28

Anchoring(2/3)

29

Anchoring(3/3)

30

ChronicleRecognition

Multi-agentsensing

32

MultiUAVTrafficMonitoring

33

DyKnowFederation

34

Cooperation

whathappenwhenanhelicopterneedtoleaveitspatrolareatogoinpursuitofanoffender?Either,needtofindareplacementforpatrollinghisareaOranotherhelicoptertopursuetheoffender

SurveillanceSequence

SurveillanceSequence

PursuitSequence

Rescueoperation

36

RescueoperationScenario(1/2)

37

RescueoperationScenario(2/2)

ExplorationtofindallvictimsDivisionofareaintoscanningareaTaskallocationDetectionofvictims

RescueofTaskallocation

HumanDetectionandApplication

39

VictimsDetection

40

Leashing

Planning

42

HowtoconstructaTask-SpecificationTree?

43

PlantoTask-SpecificationTree

AnautomatedplannertakesaproblemdescriptionObjectives

Availableresources,actions,…

…andgeneratesaplanthatachievestheobjectives

44

Naturaldisasterexample

Example:SupposetherehasbeenanaturaldisasterObjective:100peopleshouldhavefood,medicineandwaterWehaveasmallfleetofUAVsavailable

Howtodescribethistoaplanner?Howtogeneratea

45

Naturaldisasterexample:GeneralWorldDescription

High-leveldescriptionoftheworldweoperateinEntities:UAVs,crates,people,…

Properties:Entitieshavelocations,UAVshavecapabilities,…

Actionsthatcanbeperformed(:operator(deliver-crate?uav?crate?from?to)(:precond(and(has-capability?uavcarry-crates)(at?uav?from)(at?crate?from)…)(:phase:duration(flight-time?from?to):effects(:assign(location?crate)?to)(:assign(location?uav)?to)…)…)

46

Naturaldisasterexample:ProblemDescription

ThecurrentstateoftheworldLocationsofinjuredpeople,availabilityofcrates,…

Agoaltobeachieved(and(forall?person(has-crate?personfood))(has-crateperson2medicine)(has-crateperson7medicine)…)

47

Resultingplan

Theresultingplansshouldbeabletoexpress:Concurrency:sequentialplannersarenotapplicablePrecedence:uav7picksupcarrier2afteruav2loadscratesLackofprecedence:Onlywaitforotheragentswhenyouhaveto!Approximatetiming:(exactdurationsareunknown)

48

PlanningtoTask-SpecificationTree

Delegation

50

Delegation(1/2)

Missionconsistingofaflightaction+agoaltoachieve

51

Delegation(2/2)

Verifyexecutabilitythroughon-boardfunctionalitiesduringplanningMotionplannerScheduling,resourcereasoning,constraintreasoning

Infeasibleactionimmediatebacktracking!

Useofaconstraintsolver

52

DistributedPlanning

53

DelegationExample

BuildingScanning

55

BuildingScanningScenario

56

BuildingScanningpart1

57

BuildingScanningpart2

58/65

LecturesSummary

Masterthesis

60

Masterthesis

InvariousdomainofArtificialIntelligencePlanningKnowledgeRepresentationMachineLearningRoboticSensing

61

GuidedExploration

Lookingforvictims,usingpriorknowledgeaswellasnewobservation

Prioritizedscanarea

62

Objectdetectionandrecognition

Objectrecongitionusingdeeplearning

Implementationwithagroundrobot

Evaluationonaerial

63

HumanRobotInterractions

VoicecommandsPoserecognition

Andmanymore...

65/65