95
The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu Xuezhou Ma Junjun Li

The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Embed Size (px)

Citation preview

Page 1: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

The Hybrid Deliberative/Reactive Paradigm

The City College of New York

Department of Electrical Engineering

Group Member: Jik Cheung Yongwen

Zhu Yayi Hu

Xuezhou Ma Junjun Li

Page 2: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Chapter Objectives

Describe the hybrid paradigm in terms of SAP and sensing organization.Distinguish the responsibilities between the deliberative layer and reactive layer.List the basic components of a Hybrid architecture: sequencer agent, resource manager, cartographer, mission planner, performance monitoring and problem solving agent.Identify the difference between managerial, state hierarchy and model-oriented styles of Hybrid architectures.Be able to describe the use of state to define behaviors and deliberative responsibilities in state hierarchy styles of Hybrid architectures.

Page 3: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Overview

• However, the robot could not…Remember the state of the robot/worldPlan optimal trajectoriesMake mapsMonitor its own performanceSelect the best behaviors for a task

• Reactivity more art then science?

Should planning be reintroduced?

I. Reactive Paradigm is the major trend by the end of the 1980’s.

Page 4: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

II. Deliberative Vs. Planning

• Not all of these activities involve Planning:

Make maps Monitor its own performanceSelect the best behaviors for a task

• To differentiate this from path planning, the term deliberative was coined.

Page 5: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

III. Hybrids

• How can slow planning be intergraded with fast reactivity?

Five examples of architectures will be illustrated: AuRA, SFX, 3T, Saphira and TCA.

• First Opinion: The worst of both worlds!

Reactive systems for unstructured worldsHierarchical systems for knowledge-rich worlds

• Nowadays: The best of both worlds!Reactive functions for low level control

Deliberation for higher level tasks

Page 6: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Hybrid Paradigm

Organization: Plan, Sense-Act:

Page 7: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Motivation of Hybrids

• Cohesion (object oriented programming)Reactivity:

Short time horizon (Present)No global knowledgeWork with sensors and actuators

Deliberation:Long time horizon (Pass, Future)Global knowledgeWork with symbols

• Multi-taskingDeliberative functions execute in parallel with reactive functions.

Page 8: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Sensing Organization

The Map (World Model)

Can have its own sensorsCan “eavesdrop” on other sensorsCan act as “virtual” sensor

World Map/

Knowledge Rep

Behavior

Behavior

Behavior

Sensor 3

Sensor 1Sensor

2

Virtual sensor

Behavior control only

Feedback

Planning only

Eavesdrop

Page 9: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Skill Vs. Behaviors

• Not purely reflexive:Reflexive (response to stimulus)Innate (virtual sensor turns behavior on or off)

“If power is low, charge”

LearnedRetain feedback to determine best behavior sequence to instantiate next time

• More complex emergent behaviors:Behavior sequences

Page 10: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Connotations of Global

• “ Global” isn’t always truly global in Hybrids.

• Behavioral ManagementPlanning which behaviors to use requires knowledge about current and future world state

• Performance monitoringDetecting task progress and sensor confliction require knowledge about the robot hardware and the overall goals.

Nonetheless

Page 11: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Common Components

• SequencerGenerates a sequence of behaviors

• Resource ManagerAllocates resources to behaviors

• CartographerCreates, stores, maintains, accesses map information

• Mission PlannerInteract with human and create a plan to achieve a goal

• Performance Monitor/problem solverDetermines whether the robot is making progress toward its goal

Page 12: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Architecture Styles

• Managerial (division of responsibility as in business)

AuRASFX

• State Hierarchies (strictly by time scope)3T

• Model-Oriented (Model serve as virtual sensors)

SaphiraTCA

Page 13: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Styles of hybrid architectures

● Managerial styles

● State hierarchies styles

● Model-oriented styles

Page 14: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

• Managerial Architectures Description -- top agents – high level planning ↓ subordinate agents – refine plan, gather resources ↓ lowest level agents

▲ AuRA Architectures ▲ SFX Architectures

Page 15: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

▲ Autonomous Robot Architecture (AuRA)

It consists of five subsystems -- planner : responsible for mission and task planning

-- cartographer : all map making, reading functions

-- motor : motor schema

-- sensor

-- homeostatic control : modify the relationship between behaviors by changing the gain as a function of robot or other constraints

Page 16: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

AuRA Architectural Layout

Page 17: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

The table below summarizes AuRA in term of the common components and style of emergent behavior

AuRA Summary

Sequencer Agent Navigator, Pilot

Resource Manager Motor Schema Manager

Cartographer Cartographer

Mission Planner Mission Planner

Performance Monitoring Agent Pilot, Navigator, Mission Planner

Emergent Behavior

Vector summation, spreading activation of behaviors, homeostatic control

Page 18: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

▲ Sensor Fusion Effects (SFX)description – It is an extension to AuRA. The extension was to add modules to specify how sensing and handling sensor failure.

Page 19: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Deliberative layers -- Mission planner : acts as a CEO giving a directions

-- effector

-- Task

-- Sensor

All of three of above determine the best allocation of effect, sensing resource and perceptual schema.

-- Cartographer : map making, path planning

Page 20: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

SFX (Sensor Fusion Effects)

Behaviors(using direct

perception, fusion)

SenseSenseSenseSenseMuscleMuscleMuscleActuators

Deliberative Layer Managers

SenseSenseSenseSensor

SenseSenseSenseReceptiveField

Choice of behaviors, resourceallocation, motivation, context

Focus of attention,recalibration

SensorWhiteboard

BehavioralWhiteboard

Del

iber

ativ

e L

ayer

Rea

cti v

e L

a yer

Parameters to behaviors,sensor failures, task progress

actions

SuperiorColliculus-likefunctions

CerebralCortex-likefunctions

Cartographer(model/map

making)

Recognitionperception

Page 21: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Reactive layersAll these layers reflect to ------- strategic behaviors and

tactical behaviors

Tactical behavior serves as filter on strategic commands to ensure to robot acts in a safe manner in as close accordance with the strategic intent as possible

the interaction of strategic and tactical behaviors is still considered emergent behavior

Page 22: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Tactical Behaviors

sensors strategic behaviors tactical behaviors actuators

follow-path speed-controlcamera drive

motor

avoidsonar

steermotor

center-cameracamerapanmotor

inclino-meter

slope

clutter

obstacleshow much vehicle turns

direction to path safe direction

safe velocity

swivel camera

strategic velocity

Page 23: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

The table below summarizes SFX in term of the common components and style of emergent behavior

SFX Summary

Sequencer Agent Task Manager

Resource Manager Sensing and Task Manager

Cartographer Cartographer

Mission Planner Mission Planner

Performance Monitoring Agent Performance Monitor, Habitat Monitor

 Emergent Behavior Strategic behaviors grouped into abstract

behaviors or scripts, then filtered by

tactical behaviors

Page 24: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

• State-hierarchy Architectures

(3 layers)

Page 25: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

▲ 3 – tiered (3T)Used for : planetary rovers underwater vehicles robot assistants for astronauts

Page 26: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Structure -- planner : setting goal and strategic plans -- sequencer : select a set of primetive behaviors develop a task network -- skill manager : in this layer the skills have associated events to verify explicitly that an action has had to correct effect

Page 27: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

3T Architecture

Page 28: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

The table below summarizes 3T in term of the common components and style of emergent behavior

3T

Sequencer Agent Sequencer

Resource Manager Sequencer (Agenda)

Cartographer Planner

Mission Planner Planner

Performance Monitoring Agent Planner

Emergent Behavior Behaviors grouped into skills,

skills grouped into task

network

Page 29: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

• Model-oriented Architectures

two of best-known model-oriented architecture▲Saphira architecture▲Task Control Architecture

Page 30: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

▲ Saphira Architecture -- PRS-Lite it is capable of taking natural language voice commands from humans and then operationalizing that into navigation tasks and perceptual recognition routines.

-- virtual sensor

-- navigation tasks manage the behaviors

-- LPS (Local Perceptual Space) determine the planning and execution improve the quality of the robot’s overall behavior

Page 31: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Saphira Architecture

Page 32: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

The table below summarizes Saphira in term of thecommon components and style of emergent

behavior

Saphira

Sequencer Agent Topological planner, Navigation

Tasks

Resource Manager PRS-Lite

Cartographer LPS

Mission Planner PRS-Lite

Performance Monitoring Agent PRS-Lite

Emergent Behavior Behaviors fused with fuzzy logic

Page 33: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

▲ Task Control Architecture (TCA) -- Task Scheduling (Mission Planner) determine the goal and order of execution

-- Path Planning (Cartographer)

-- Navigation (Sequencer) to determine what the robot should be looking for, where it is, where it has been.

-- Obstacle Avoidance To factor in not only obstacle but how to respond with a smooth trajectory for the robot’s current velocity.

Page 34: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

TCA

Page 35: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

The table below summarizes TCA in term of the common components and style of emergent behavior

TCA

Sequencer Agent Navigation Layer

Resource Manager Navigation Layer

Cartographer Path-Planning Layer

Mission Planner Task Scheduling Layer

Performance Monitoring Agent

Navigation, Path-Planning, Task-Scheduling

Emergent Behavior Filtering

Page 36: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Basic Important concept

• ParadigmParadigm is both a way of looking at the world and an implied set of tools for solving problems.

• Sense, Plan, Act. Commonly accepted robotic primitives.Robotics have to go through these three, or at least two process to complete a mission.

• Local Processing and Global World ModelLocal: sensor data used in specific for each function.Global: all sensor data is processed to single model.

Page 37: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Hierarchical Paradigm

• What are the two main features?Robot operates in a top-down fashion.All sensor data tends to be gathered to one global world model. A single representation that planner can use to rout the action.

SENSE PLAN ACT

Page 38: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Reactive Paradigm

• What are the two main features?Throw out planning all together.The inputs to an act are the direct output of a sensors.examine living example of intelligence.

SENSE ACT

Page 39: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Hybrid Paradigm

• Features of Hybrid Deliberative/Reactive Paradigm

It is reactive planning, Planning to subtask is done at one step.Deliberative planning take a long time comparing to the time of reactive executionSensor data go directly to each behavior but is also available to the planner for construction of task-oriented global world model.Model-based Architecture focuses on the creation and maintenance of a global world model.

Page 40: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Hybrid Paradigm

• The basic models of Hybrid ParadigmSequencer: generate a set of behaviors for subtasks.Resource manger: allocate resources to behaviorCartographer: for creating, storing, maintaining map or spatial information.Mission Planner: interact with man, construct a mission plan.Performance Monitoring: monitor the process of the executing, It’s self-awareness.

Page 41: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Hybrid Paradigm

ACTSENSE

Plan

Page 42: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Hybrid Paradigm

Robot Primitive

Input output

PLAN Directives

BEHAVIOR

Information( sensed and

cognitive )

Sensed data Actuator command

Page 43: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Other Hybrid Paradigm

• DARPA UGV Demo II and Demo III. Outdoor ground vehicle control and navigation. given a map and a set of directions find enemy location.Reach in automating highway vehicles by European Community ESPRIT agency and some United States agencyAutonomous planetary rovers by NASA. Mapping planetary surface, planning path.

Page 44: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Advantages of Hybrid Architecture is highly modular

Architecture is highly modular of the deliberative with object-oriented programming.

• Full knowledge of environmentSoftware agents can use agent-specific abstractions to exploit the structure of an environment in order to fulfill their particular role in deliberation.

• Use of Global modelsGlobal models are only for symbolic functions and Planners( sequencers) often produce partial plans.

Page 45: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Advantages of Hybrid

• Execution is reactive.• No frame problems.

In the Hybrid Paradigm almost no the frame problems resulted by the Hierarchical.

• Self-consciousness.Ensure robustness by monitoring the performance of the robot and self-diagnosing, this is called self-consciousness.

Page 46: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Examples For Good of The Reactive

• Example1 we don’t need to turn all sensed data to global model to use in order to accuracy, convince, reliability, and saving time.

• Example 2 in Hierarchical Paradigm it is unwise in a lot of practical problems to block out the sensed data to Behaviors( Actuator).

Page 47: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

II. f

LED

Sensor 1

Sensor 2

Sensor 3 Pressure Sensor

A/D D/A

A/DCPU

Page 48: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

GasSens

or 1

Alarm

CPU

A/D D/A

D/AA/D

Page 49: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Interleaving Deliberation and Reactive Control

• For navigationDeliberation: Cartographer( planner) generates a complete optimal route, decompose the route to segments-waypoints.Reactive Control: Waypoint can be accomplished by behaviors.

• Top-down methodDeliberative layers decompose the missions to

finer steps. Reactive layers accomplish the first sub-goal.

Page 50: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

• Bottom-up method.Deliberative layers act as virtual sensors. The analyzed information as a sensed data input into behaviors( reactive layers)-Bottom-up

• Other functions of DeliberationsIn the deliberative layers, sequencer must know why a failure and know the need to change the behaviors and alert the human supervisor.-self-consciousness.

Interleaving Deliberation and Reactive Control

Page 51: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Summary of AI Robotics

Page 52: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

• What is intelligent robots?• What is the difference between AI and

Engineering approaches to robotics?• What is the difference between

telepresence and semi-autonomous control?

Ch.1: From Teleoperation to Autonomy

Page 53: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

What is intelligent robots?

• Mechanical creatures that can function autonomously, which means it can sense, act, maybe even reason; doesn’t just do the same thing over and over like automation.

• The intelligent robots arose by the development of AI since the 1990’s.

Page 54: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Teleoperation

• Teleoperation is that a human operator controls a robot from a distance.

• It is a ideal solution for controlling remotes because AI technology is nowhere near human levels of competence, especially in terms of perception and decision making.

• Cons: Cognitive fatigues; communications dropout; communications bandwidth; communications lag;

Page 55: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Add more intelligence to the early teleoperation

• Telepresence – providing sensory feedback to the point that

teleoperator feels they are “present” in robot’s environment by adding more cameras.

• Semi-autonomous control– human is involved, but routine or “safe” portions of

the task are handled autonomously by the robot– It is really a type of mixed-initiative

Page 56: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

The Seven Areas of AI

• knowledge representation– How does the robot represent its world, task, and itself.

• understanding natural language– Natural language is usually challenging, it is not only

talking about looking up words from a dictionary by understanding.

• Learning– A robot could be programmed by just watching a

human’s behaviors.

Page 57: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

The Seven Areas of AI

• planning and problem solving– The ability to plan actions and solve problems with

those plans• Inference

– Inference is generating an answer when there is no complete information

• Search– Search means efficiently examining a knowledge

representation of a problem to find the answer.• Vision

– The robot can simulate the effects of actions in its “head”

Page 58: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Robotics Paradigms

• What are robotic paradigms?– A paradigm is a philosophy or set of

assumption and/or techniques which characterize an approach to a class of problems.

• There paradigms:– Hierarchical paradigm (Ch. 2)– Reactive paradigm (Ch. 4)– Hybrid paradigm (Ch. 7)

Page 59: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Ch. 2: Hierarchical paradigm

• The oldest paradigm, and was prevalent from 1967-1990.

• Under this paradigm, the robot senses the world, plans the next action, and then acts.

PLANSENSE ACT

Page 60: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Strips: means-ends analysis

• Strips is a variant of the general problem solver method, it uses an approach of means-ends analysis, where if the robot can’t accomplish the task in one “movement”, it picks a action which will reduce the difference between what the now state versus the goal state.

• To implement Strips, Designer must set up– World model representation– Difference table with operators, preconditions, add & delete lists– Difference evaluator

Page 61: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Strips: means-ends analysis

• Strips assumes closed world– Closed world: world model contains

everything needed for robot (implication is that it doesn’t change)

– Open world: world is dynamic and world model may not be complete

• Strips suffers from frame problem– Frame problem: representation grows too

large to reasonably operate over

Page 62: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Representative Architecture

• An architecture is a method of implementing a paradigm, of embodying the principles in some concrete way.

• The two best known architectures are the Nested Hierarchical Controller (NHC) developed by Meystel and the NIST Realtime Control System (RCS) originally developed by Albus.

Page 63: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

• support for modularity:– decomposition by functionality

• niche targetability: – good, both have been used for apps like vehicle guidance, mining

equipment

• ease of portability to other domains: – unclear, not sure if code could be reused—lots of rewriting on

previous apps

• robustness:– RCA simulates plans in advance, but not sure what it would do with

sensor or mechanical failures, etc.

Evaluating the Two Architectures

Page 64: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Advantages and Disadvantages

• Advantages:– It provides an ordering of the relationship

between sensing, planning, and acting.

• Disadvantages:– Planning: for every update cycle, robots had to

do some type of planning.– Dependence on a global world model– Uncertainty: did the robots actually finish the

action? We don’t know for sure.

Page 65: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Ch. 3: Biological Foundations of the Reactive Paradigm

• Why explore the biological sciences?• What are the three levels in a

computational theory?• What are animal behaviors?• Coordination of behaviors, perception,

schema theory, and more…

Page 66: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Why do we need to explore the biological sciences?

• Animals and man provide existence proofs of different aspects of intelligence.

• The principles of animal intelligence are extremely important.– For examples: roboticists may overcome the

closed world assumption that presented problems with shakey by observing the animals behaviors in an open world.

Page 67: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Marr’s Computational Theory

• The levels in the computation theory can be stated as:

• Level 1: What is the phenomena we’re trying to represent?

• Level 2: How it be represented as a process with inputs/outputs?

• Level 3: How is it implemented?

Page 68: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Animal Behaviors

• A behavior is a mapping of sensory inputs to a pattern of motor actions which then are used to finish a task

• Three catagories:– Reflexive

• stimulus-response, often abbreviated S-R– Reactive

• learned or “muscle memory”– Conscious

• deliberately stringing together

Page 69: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Coordination and Control of Behaviors

• There are four ways to acquire a behavior, which are:• To be born with a behavior (innate)

– Examples: Arctic terns.• To be born with a sequence of innate behaviors.

– Examples: mating cycle in digger wasps.• To be born with behaviors that need some initialization

(innate with memory).– Examples: bees, which are born with in hives.

• To learn a set of behaviors– Examples: Lions, who are nor born with any hunting

behaviors.

Page 70: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

How behaviors are coordinated and controlled-- innate releasing mechanisms (IRM)

BEHAVIOR

SensoryInput

Patternof MotorActions

Releaser

• The Releaser acts as a control signal to activate a behavior. If a behavior is not released, it does not respond to sensory inputs.

Page 71: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Perception

• Two functions of perception (can be the same percept)– Release a behavior– Guide a behavior

• Action-oriented perception (Neisser)– Planning is not needed to act – Perception is selective

CognitiveActivity

World

Perceptionof

Environment

Samples, FindsPotential Actions

Acts &ModifiesWorld

Directs whatto look for

Page 72: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Schema Theory

• Schema theory provides a helpful way of casting some of the insights from above into an OOP format.

• is generic, equivalent to an object in OOP– schema specific knowledge (local data)– procedural knowledge (methods)

• schema intiantation is specific to a situation, equivalent to an instance in OOP

• a behavior is a schema, consists of– perceptual schema– motor schema

Page 73: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Ch. 3: Summary

• A behavior is the fundamental element of biological intelligence, and will server as the fundamental component of intelligence in most robot systems.

• Innate Releasing Mechanisms (IRM) are one model of how intelligence is organized.

• Perception in behaviors serves two roles, including a releaser for a behavior and a precept which guides the behavior.

• Schema theory is an object-oriented way of representing and thinking about behaviors.

Page 74: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Ch. 4: The Reactive Paradigm

• The Reactive Paradigm was a reaction to the Hierarchical Paradigm, and it was heavily used between 1988-1992.

• The fast execution time can be achieved by throwing away “Planning”.

SENSE ACT

RELEASER behavior

Page 75: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Reactive Robots

• Most apps are programmed with this paradigm• Biologically based:

– Behaviors (independent processes), released by perceptual or internal events (state)

– No world models or long term memory– Highly modular, generic– Overall behavior emerges

SENSE ACT

RELEASERbehavior

Page 76: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Hierarchical Organization is“Horizontal”

• Horizontal decomposition of tasks into the S, P, A organization of the Hierarchical Paradigm.

Page 77: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

More Biological is “Vertical”

• The right figure shows that a vertical decomposition of tasks into an S-A orgrnization.

Page 78: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Architectures

• Historically, there are two main styles of creating a reactive system:– Subsumption architecture

• Layers of behavioral competence• How to control relationships

– Potential fields• Concurrent behaviors• How to navigate

Page 79: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Subsumption Architecture

• Subsumption has a loose definition of behavior as a tight coupling of sensing and acting.

• Higher layes may subsume and inhibit behaviors in lower layers.

• The design of layers and their behaviors is usually difficult.• Behaviors are released by the presence of stimulus.• Subsumption solves the frame problem by eliminating the

need to model the world because the behaviors just simply respond to whatever stimulus is in the environment.

• Perception is largely direct, using affordances.• Perception is ego-centric and distributed.

Page 80: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Potential Fields

• Potential field styles of behaviors always use vectors to represent behaviors and vector summation to combine vectors from different behaviors to produce an emergent behavior.

• Behaviors are defined as consisting of one or more of both motor and perceptual schemas and (or) behaviors.

• All behaviors operate concurrently and output vectors are summed.

• Behaviors may make varying contributions to the overall action of the robot, although they are treated equally.

• Perception is usually handled by direct perception or affordances.

• Perception can be shared by multiple behaviors.

Page 81: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Evaluation of Reactive Architectures

• Support for modularity– Both decompose the actions and perceptions. Subsumption

favors a composition suited for a hardware implementation, whereas potential fields methods for a software-oriented system.

• Niche targetability– Both have hign targetabilities.

• Ease of portability to other domains– Subsumption depends on low layers heavily, while potential

fields usually have no implicit reliance on a low layer.• Robustness

– Neither can be called genuinely robust.

Page 82: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Ch. 4: Summary

• The organization of the Reactive Poradigm is SENSE-ACT, No PLAN component.

• Under reactive paradigm, behaviors serve as the basic building blocks for robot actions.

• Reactive systems also exhibit good software engineering principles due to the programming by behavior approach.

• At last, two representative architectures are subsumption and potential fields. However, despite the differences in theory, these two systems appear to be largely equivalent practically.

Page 83: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

The key points to understand what is main characters of

AI robotics?

OOP (Object-Oriented Programming)Model of sensingHybrid deliberative/Reactive ParadigmExample of our homework#3Future of Robot

Page 84: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

What is OOP? Object-Oriented Concepts tap into this natural

human tendency resulting in an easy to understand and use language.

An automobile is a very good example of the Object-Oriented Concept. As humans, it is our natural tendency to think of an automobile as a single "thing", and not as a large group of several thousand small "things". Thinking of the automobile as a single "thing" helps us deal with the overwhelming complexity of the whole machine. We would say simple statements like; "Fill her up.“ or "How fast are we going?" or "I have a Blue car. " ..... and everyone would understand how those statements apply to our

car.

Page 85: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

1. Example for OOP Programming

Using an automobile as an example of an Object, the following

program shows an example of Object Oriented programming: BobsCar.Speed = 50 If BobsCar.Speed>CurrentRoad.SpeedLimit Then PoliceCar.Mode = Chase PoliceCar.Target = BobsCar PoliceCar.Speed = BobsCar.Speed + 10 End If Is it very simple and easy to understand? Here, please imagine that if we do not use OOP, what should

our program look like?

Page 86: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

2. How behaviors can be implemented using OOP constructs such as classes? Recall from software engineering that an object consists of data and method, also called attributes and operations. And as noted before, schemas contain specific knowledge and local data structures and other schemas. So, a schema as a programming object will be a class. It’s defined as below:

Page 87: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

3. Example: move-to-go behavior

1) We put a robot in an empty arena with Coca-cola cans in random location and a blue recycling bin in a corner.

2) The behaviors needed is picking up a red can and moving to a blue bin. But we write a single generic behavior move_to_goal (color) to deal with both behaviors.

3)The behavior move_to_goal consist of a perceptual schema, which will be called extract-goal and a motor schema, which used an attractive field. extract-goal uses the affordance of color to extract where the goal is in the image, and then computer the angle to the center of the colored region and size of the region.

The table below implies some important points about programming with behaviors:

Page 88: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Object Behavioral Analog

Identifier

Data Percept goal_anglegoal_strength

Method

Perceptual_schemaMotor_schema

extract_goal(goal_color)Pfield.attraction(goal_angle, goal_color)

4) The attraction motor schema takes that

percept and is responsible for using it to

turn the robot to center on the region and

move forward.

5) Two schemas are both independent. The perceptual

schema doesn’t know the existence of motor schema.

Page 89: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

1. Model of sensing

environment

Sensor

ObservationOr Image

Perceptual

Schema

MotorSchema

Robot Action

Percept

Sensor/transducer---------->Behavior------------->Action

Page 90: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

2. Behavioral Sensor Fusion

Sensor

Sensor

Sensor

Fusion Behavior

Perception in a reactive robot system has two roles:

1)to release a behavior 2)to support or guide the action of the

behavior All sensing is behavior-specific, where

behaviors map tap into the same sensors, but use the

data independently of each other.

Page 91: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

The Hybrid Deliberative /Reactive Paradigm

1. It can be thought as PLAN, then SENSE—ACT.

2. The SENCE—ACT portion is always done with reactive behaviors, where PLAN includes a

broader range of intelligent activities.3. Planning can be interviewed with execution.4. Architecture usually encapsulate functionality

into modules. The basic modules are: mission

planner, behavior manager, performance monitor.5. State-hierarchies divide deliberation and

reaction by the state, available to the modules or agents

operating that layer. Three states are: Past, Present,

Future.

Page 92: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Example

Plan (the Algorithm we use)ś=f (s(i));δ=g (Ψ(s), Ψd(s));s(i+1)=h (s(i));Xd=f1(s(i)); Yd=f2(s(i))

Sense (Virtual Vehicle) Xd(s), Yd(s), Ψ(s), Ψd(s)

ACT (Actual Robot)X(s+1), Y(s+1), Ψ(s+1), Ψd(s+1)

Do we use PLAN—SENSE—ACT concept? Modules concept? State-hierarchies? Planning can be interviewed with execution?

Page 93: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Future of Robot

Enabling technologies Enabling technologies ranging from sensors to

radio communications and navigation aids are all

accelerating logarithmically. The ubiquitous acceptance of

wireless LAN systems, the plunging costs of video

cameras and processors, the availability of affordable laser

navigation systems, and the ever-increasing accuracy and

dropping cost of GPS navigation receivers are all

combining to make autonomous robots potentially cheaper and

ever more capable.

At least as important, we now have enormous resources

Page 94: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

in human experience. Countless software engineers and academics have spent endless hours developing concepts of modeling and control that are just as much part of the existing

robotics toolbox as any sensor or processor. As a result, only the

integration of these elements is required for new robotic configurations to burst

onto the scene with blinding speed.

Social forces The social issues already discussed are pushing customers to look

for new solutions to performing many of the tasks that now require

manual labor. These are tasks which autonomous robots can easily provide.

Slowly but surely, a few venture capitalists (real ones) are beginning to

make investments in companies like iRobot, and the industry is

beginning to gain a little attention.

Page 95: The Hybrid Deliberative/Reactive Paradigm The City College of New York Department of Electrical Engineering Group Member: Jik Cheung Yongwen Zhu Yayi Hu

Thank you for your time!