Lecture 2: Intelligent Agents

Heshaam Failihfaili@ece.ut.ac.ir

University of Tehran

What is an intelligent agent?

Structure of intelligent agents

EnvironmentsExamples

2

Intelligent agents: their environment and actions

3

Concepts Environment Sensor Actuator( effectors ) Agent Functions Agent Program Percept and Percept Sequence Rational Agent Performance measure

4

Ideal rational agents For each possible percept sequence, an ideal

rational agent should take the action that is expected to maximize its performance measure, based on evidence from the percept sequence and its built-in knowledge.

Key concept: mapping from perceptions to actions

Different architectures to realize the mapping

5

Structure of intelligent agents

Agent program: a program that implements the mapping from percepts to actions: The JOB of AI

Architecture: the platform to run the program (note: not necessarily the hardware!)

Agent = architecture + program Examples:

medical diagnosis- part-picking robot satellite image analysis - interactive tutor refinery controller - flight simulator

6

An Example(Vacuum Cleaner)

What is good Performance measure?

amount of dirt cleaned up in a single eight-hour shift ??

…Clean and dump it again, …

reward the agent for having a clean floor.

For example, one point could be awarded for each clean square at

each time step

As a general rule, it is better to design performance

measures according to what one actually wants in the environment, rather than

according to how one thinks the agent should behave.

7

Task Environment PEAS: Performance, Environment,

Actuator, Sensors

8

More Examples

9

Properties of environments

Fully Observable or Partially Observable: is the state of the world fully know at each step?

Deterministic to Stochastic: how much is the next state determined by the current state? If the environment is deterministic except for

the actions of other agents, we say that the environment is strategic.

Episodic to sequential: how much state memory?

10

Properties of environments

Static to dynamic: how much independent change? SemiDynamic: If the environment itself does

not change with the passage of time but the agent's performance score does

Discrete to continuous: how clearly are the actions and percepts differentiated?

Single Agent or multi Agent: Competitive or Co-operative

11

Examples of environments

12

Rational or OmniScience An omniscient agent knows the actual

outcome of its actions and can act accordingly; but omniscience is impossible in reality.

Rationality is not the same as perfection Rationality maximizes expected

performance, while perfection maximizes actual performance.

13

Table-Driven Agentsfunction Table-driven-agent(percept) returns action

static: percepts, a sequence, initially empty

table, indexed by percept sequences (given)

append percept to the end of percepts

action := LOOKUP(percepts, table)

return action

• Keeps a list of all percepts seen so far• Table too large• takes too long to build• might not be available

TAXI DRIVER: need table with 10 250,000,000,000 entries for 1 hour driving

(30frames per second 640*480 * 24 color)CHESS: 10 150

14

Simple Reflex Agent (1)e.g.

if car-in-front-is-braking then initiate-braking.

15

Simple Reflex Agent (2)function Simple-Reflex-Agent(percept) returns action

static: rules, a set of condition-action rules

state := Interpret-Input (percept)

rule := Rule-Match(state, rule)

action := Rule-Action[rule]

return action

• No memory, no planning

16

Simple Reflex example

17

Partially Observable Simple Reflex Vacuum cleaner with only one

sensor: Only get Status (Dirty/Clean) Infinite loops unavoidable in partially

observable Randomized Simple Reflex

18

Model-based Reflex Agents The most effective way to handle

partial observability is to keep track of the part of the world it can't see now.

Store some of internal nodes Change internal nodes based on

Environments and Agent Actions Model the Environment by some nodes

19

Model-based Reflex Agents

20

Model-based Reflex Agents

function Reflex-Agent-State(percept) returns action

static: rules, a set of condition-action rules

state, a description of the current state

state := Update-State (state, percept)

rule := Rule-Match(state, rules)

action := Rule-Action[rule]

state := Update-State (state, action)

return action• still no longer-term planning

21

Goal-based Agents Knowing the current state is not

always enough to decide what to do. For example, the taxi can turn left,

turn right, or go straight on. The correct decision depends on where the taxi is trying to get to.

GOAL-BASED

22

Goal-based Agents (1)

23

Goal-based Agents (2)function Goal-Based-Agent(percept, goal) returns action

static: rules, a set of condition-action rules

state, a description of the current state

state := Update-State (state, percept)

rule := Plan-Best-Move(state, rules, goal)

action := Rule-Action[rule]

state := Update-State (state, action)

return action

• longer term planning, but what about cost?

24

Utility-based Agents (1) Goals alone are not enough to generate high-

quality behavior There are many action sequences that will get the

taxi to its destination Add utility evaluation: not only how close

does the action take me to the goal, but also how useful it is for the agent

Other aspects to be considered: uncertainty in perceptions and actions incomplete knowledge environment characteristics

25

Utility-based Agents (2)

26

Learning Agent How the agent programs come into being? Turing (1950) considers the idea of

actually programming his intelligent machines by hand.

But he also proposed to build learning machines and then to teach them (Learning).

Learning allows the agent to operate in initially unknown environments and to become better and better…

27

Learning Agent - Learning element: responsible for making

improvements-Performance elements: responsible for

selecting external actions- critic : learning elements uses feedback

from Critic-Problem generator: It is responsible for

suggesting actions that will lead to new and informative experiences.

28

Learning Agent The learning element can make

changes to any of the "knowledge" components shown in the agent diagrams learning directly from the percept sequence. Observation of pairs of successive states

("How the world evolves“) observation of the results of its

actions ("What my actions do.“)

29

?