Advanced Mobile RoboticAdvanced Mobile Robotic The City College of the City University The City College of the City University

of New Yorkof New York

Introduction to AI Robotics

Instructor: Prof. Jizhong XiaoPresenters: Diana Acevedo Sefton Bennett

Clara Nieto-Wire Jorge O. Peche

Marios Timotheou

March 13, 2006March 13, 2006

Reactive ParadigmReactive Paradigm

Biological foundations of the reaction Biological foundations of the reaction paradigm paradigm

The reactive paradigmThe reactive paradigm Designing a reactive implementation Designing a reactive implementation

Chapter 3Chapter 3

Biological foundations of Biological foundations of the reaction paradigmthe reaction paradigm

Why??????Why??????

Animals Animals Open World Open World

AGENT AGENT Self-Contained and IndependentSelf-Contained and Independent

Interacts with the world to Interacts with the world to make changes or sense what is make changes or sense what is happening.happening.

Humans-Animals-Robots = AGENTHumans-Animals-Robots = AGENT

Marr’s Computational TheoryMarr’s Computational Theory

Level 1

What are we trying to represent?

Level 2

How Represent it?

Level 3

How is it implemented?

Computational Theory LevelsComputational Theory Levels

1.1. Existent proof of what can/should be done. Existent proof of what can/should be done. Agents share commonality of purpose or Agents share commonality of purpose or functionality.functionality.

2.2. ““What”=Inputs, outputs and transformations. What”=Inputs, outputs and transformations. Creating Flow Charts of black boxes. Agents Creating Flow Charts of black boxes. Agents exhibit common processes.exhibit common processes.

3.3. How to implement the processes. Agents may How to implement the processes. Agents may have little or not commonality in their have little or not commonality in their implementations. implementations.

BEHAVIORBEHAVIOR

Mapping of sensory inputs to a pattern of Mapping of sensory inputs to a pattern of motor actions which then are used to motor actions which then are used to achieve a task.achieve a task.

Pattern Pattern Actions and sequences that are Actions and sequences that are always the same.always the same.

BEHAVIORBEHAVIORSENSORY SENSORY INPUTINPUT

PATTERN OF PATTERN OF MOTOR ACTIONSMOTOR ACTIONS

Types of BehaviorTypes of Behavior

ReflexiveReflexive : Stimulus-Response.: Stimulus-Response.

ReactiveReactive : Learn-Consolidate: Learn-Consolidate

ConsciousConscious : Deliberative: Deliberative

Reactive Paradigm Reactive Paradigm Reflexive Behavior Reflexive Behavior

Reflexive BehaviorReflexive Behavior

NO CONGNITIONNO CONGNITION

Reflexes Reflexes : T: TRESPONSERESPONSE = T = TSTIMULUSSTIMULUS

TaxesTaxes : Response : Response Movement Movement

Fixed-Action Patterns:Fixed-Action Patterns: TTRESPONSERESPONSE > T > TSTIMULUSSTIMULUS

……Ways to Acquire BehaviorsWays to Acquire Behaviors

InnateInnate

Sequence of innate Sequence of innate behaviorsbehaviors

Innate with Innate with memorymemory

LearnLearn

Innate Releaser MechanismsInnate Releaser Mechanisms

Releaser: Control Signal Releaser: Control Signal

(internal or external).(internal or external). Behaviors can be implicitly chained Behaviors can be implicitly chained

together by their releasers.together by their releasers.

IRM IRM Activates Activates the Behaviorthe Behavior

BEHAVIORBEHAVIORSENSORY SENSORY INPUTINPUT

PATTERN OF PATTERN OF MOTOR ACTIONSMOTOR ACTIONS

Concurrent BehaviorsConcurrent Behaviors

Equilibrium Equilibrium Behaviors balance Behaviors balance

each each other.other.

Dominance of one Dominance of one Winner takes all.Winner takes all.

Cancellation Cancellation Behaviors cancel each Behaviors cancel each other.other.

PERCEPTIONPERCEPTION

Cognitive activities Cognitive activities (Feedback and Feed forward control)(Feedback and Feed forward control)

• PlanningPlanning• ReactingReacting

Two uses of perceptionTwo uses of perception• Release a behaviorRelease a behavior• Guide a behaviorGuide a behavior

WORLDWORLD

Samples, FindsSamples, FindsPotential ActionsPotential Actions

Acts &Acts &ModifiesModifiesWorldWorld

Directs whatDirects whatto look forto look forCOGNITIVECOGNITIVE

ACTIVITYACTIVITY

PERCEPTIONPERCEPTIONOFOF

ENVIROMENTENVIROMENT

Gibson’s Ecological ApproachGibson’s Ecological Approach

Affordances : Perceivable potentialities Affordances : Perceivable potentialities of of the environment for an action.the environment for an action.

• Directly Perceivable.Directly Perceivable.• Sensing does not need/require memory or Sensing does not need/require memory or

interpretation (optic flow).interpretation (optic flow).

Structural models attempt to describe an Structural models attempt to describe an object in terms of physical components.object in terms of physical components.

Neisser: Two perceptual systemsNeisser: Two perceptual systems

Direct Perception: Direct Perception: Gibsonian or Gibsonian or ecological track of the ecological track of the brain. Structures low brain. Structures low in the brain that in the brain that evolved earlier. evolved earlier. (Affordances)(Affordances)

Recognition: More Recognition: More recent perceptual recent perceptual track in the brain. track in the brain.

Schema TheorySchema Theory

SchemaSchema Knowledge of how to act Knowledge of how to act and/or perceive. and/or perceive.

Computational process Computational process to to accomplish the activity. accomplish the activity.

• schema instantationschema instantation is specific to a is specific to a situation, equivalent to an situation, equivalent to an instance in instance in OOP.OOP.

Chapter 4Chapter 4

The Reactive ParadigmThe Reactive Paradigm

ObjectivesObjectives Define what the reactive paradigm is in terms of :Define what the reactive paradigm is in terms of :

• i) the three primitives SENSE, PLAN, and ACTi) the three primitives SENSE, PLAN, and ACT• ii) sensing organizationii) sensing organization..

List and discuss the characteristics and connotations of a reactive List and discuss the characteristics and connotations of a reactive robotic systemrobotic system..

Describe the two dominant methods for combining behaviors in a Describe the two dominant methods for combining behaviors in a reactive architecture.reactive architecture.• i) subsumptioni) subsumption• ii) potential field summation.ii) potential field summation.

Evaluate these two reactive architectures in terms of:Evaluate these two reactive architectures in terms of:• i) support of modularityi) support of modularity• ii) niche targetabilityii) niche targetability• iii) ease of portability to other domains iii) ease of portability to other domains • iv) robustnessiv) robustness

Be able to program a behavior using a p.field methodologyBe able to program a behavior using a p.field methodology..

Be able to construct a new p.field from primitive p.fields, and sum Be able to construct a new p.field from primitive p.fields, and sum the p.fields to generate and emergent behavior.the p.fields to generate and emergent behavior.

is characterized by having a is characterized by having a horizontal decompositionhorizontal decomposition..

is characterized by having a is characterized by having a vertical decompositionvertical decomposition..

If anything happens to an advanced If anything happens to an advanced behavior, the lower behaviors would behavior, the lower behaviors would still operate.still operate.

All reactive systems are composed of All reactive systems are composed of behaviorsbehaviors..

HierarchicalHierarchical ParadigmParadigm

Reactive ParadigmReactive Paradigm

Attributes of Reactive ParadigmAttributes of Reactive Paradigm

Behaviors.Behaviors.

SENSE-ACT organization.SENSE-ACT organization.

Behavior specific (local) sensing.Behavior specific (local) sensing.

BEHAVIORBEHAVIOR

(Ethological def.) a direct mapping of sensory inputs to a pattern of (Ethological def.) a direct mapping of sensory inputs to a pattern of motor actions that are then used to achieve a task.motor actions that are then used to achieve a task.

(Mathematically) a transfer function , transforming sensory inputs into (Mathematically) a transfer function , transforming sensory inputs into actuator commands.actuator commands.

SchemaSchema: : a way of expressing the basic unit of activity. It is consists of the knowledge of how to a way of expressing the basic unit of activity. It is consists of the knowledge of how to act or perceive, and the algorithm by which it uses to accomplish the activity.act or perceive, and the algorithm by which it uses to accomplish the activity.(like a generic template for doing some activity).(like a generic template for doing some activity).

Attributes of Reactive ParadigmAttributes of Reactive Paradigm

SENSE-ACT Organization (S-A)SENSE-ACT Organization (S-A)

The S-A organization does not specify how the The S-A organization does not specify how the behaviors are coordinated. behaviors are coordinated.

The SENSE and ACT The SENSE and ACT components are tightly components are tightly coupled into behaviors coupled into behaviors and all robotic and all robotic activities emerge as activities emerge as the result of these the result of these behaviors operating behaviors operating either in sequence or either in sequence or concurrently. (NO concurrently. (NO PLAN component).PLAN component).

Attributes of Reactive ParadigmAttributes of Reactive Paradigm

What happens when multiple What happens when multiple behaviors are active behaviors are active

simultaneously?simultaneously?

Attributes of Reactive ParadigmAttributes of Reactive Paradigm

Behavior-Specific (local) SensingBehavior-Specific (local) Sensing

•Sensing is localSensing is local

•Sensors can be Sensors can be sharedshared

•Sensors can be Sensors can be fused locally by a fused locally by a behaviorbehavior

Behaviors favor the use of affordances (direct perceptions). The behavior does not rely on any Behaviors favor the use of affordances (direct perceptions). The behavior does not rely on any central representation built up from all sensors.central representation built up from all sensors. The sensing portion of the behavior is The sensing portion of the behavior is

nearly instantaneous and action is very rapid. nearly instantaneous and action is very rapid.

Connotations of Reactive Behaviors:Connotations of Reactive Behaviors:

A reactive robotic system executes A reactive robotic system executes rapidly.rapidly.

Behaviors can be implemented directly in hardware as Behaviors can be implemented directly in hardware as circuits or with low computational complexity.circuits or with low computational complexity.

Reactive robotic systems have no memoryReactive robotic systems have no memory Reactive behaviors are limited toReactive behaviors are limited to pure “stimulus-response pure “stimulus-response

reflexes”reflexes”

BEHAVIORS ARE CONTROLLED BY WHAT IS HAPPENING IN THE WORLD, BEHAVIORS ARE CONTROLLED BY WHAT IS HAPPENING IN THE WORLD, DUPLICATING THE SPIRIT OF INNATE RELEASING MECHANISMS, rather DUPLICATING THE SPIRIT OF INNATE RELEASING MECHANISMS, rather than by the program storing and remembering what the robot did lastthan by the program storing and remembering what the robot did last . .

Characteristics of Reactive Behaviors:Characteristics of Reactive Behaviors:

Robots are situated agents Robots are situated agents (integral part of the world)(integral part of the world) operating in operating in an ecological niche an ecological niche (goals of robot, world where it operates, and how it (goals of robot, world where it operates, and how it perceives the world).perceives the world).

Behaviors serve as the building blocks for robotic actions. Behaviors serve as the building blocks for robotic actions. And the overall behavior of the robot is emergent.And the overall behavior of the robot is emergent.

Only local, behavior-specific sensing is permitted. Only local, behavior-specific sensing is permitted. (Any sensing which does require representation is expressed (Any sensing which does require representation is expressed

in ego-centric coordinates).in ego-centric coordinates).

These systems inherently follow good software design These systems inherently follow good software design principles.principles.

Animals models of behavior are often cited as a basis for Animals models of behavior are often cited as a basis for these systems or a particular behavior.these systems or a particular behavior.

Advantages of programming by Advantages of programming by behavior:behavior:

Behaviors support good software Behaviors support good software engineering principles through:engineering principles through:

decompositiondecomposition modularity and modularity and incremental testing.incremental testing.

Representative ArchitecturesRepresentative Architectures

Potential FieldPotential Field Methodologies Methodologies

SubsumptionSubsumption Architecture Architecture

Provide mechanisms for:Provide mechanisms for: Determine what happens when multiple behaviors are Determine what happens when multiple behaviors are

active at the same timeactive at the same time Triggering behaviorsTriggering behaviors

Subsumption ArchitectureSubsumption Architecture

Subsumption refers to how behaviors are combined.Subsumption refers to how behaviors are combined.In many applications behaviors are embedded directly in the In many applications behaviors are embedded directly in the hardware or on small microprocessors. hardware or on small microprocessors.

Subsumption PhilosophySubsumption Philosophy Modules should be grouped into Modules should be grouped into

layers of competencelayers of competence

Modules in a higher lever can Modules in a higher lever can override or override or subsumesubsume behaviors in the behaviors in the next lower levelnext lower level• Suppression: substitute input Suppression: substitute input

going to a modulegoing to a module• Inhibit: turn off output from a Inhibit: turn off output from a

modulemodule

No internal stateNo internal state in the sense of a in the sense of a local, persistent representation local, persistent representation similar to a world model. similar to a world model.

Architecture should be Architecture should be taskabletaskable: : accomplished by a higher level accomplished by a higher level turning on/off lower layersturning on/off lower layers

Subsumption ArchitectureSubsumption ArchitectureEXAMPLE EXAMPLE •Level 0 Avoid collisionsLevel 0 Avoid collisions

Robot-centric polar Robot-centric polar plot of 8 sonar plot of 8 sonar

range readings range readings

Polar plot unrolled into a plotPolar plot unrolled into a plot

Vector representation of sonar readingsVector representation of sonar readings

Level 0 recast as primitive Level 0 recast as primitive behaviorsbehaviors

EXAMPLE LEVEL 1: WANDEREXAMPLE LEVEL 1: WANDER

Level 1: WanderLevel 1: WanderLevel 0 : Avoid Collisions Level 0 : Avoid Collisions

•InhibitionInhibition•SuppressionSuppression

Subsumption ArchitectureSubsumption Architecture

Level 1 recast as primitive behaviorsLevel 1 recast as primitive behaviors

Subsumption ArchitectureSubsumption ArchitectureEXAMPLE LEVEL 2: Follow CorridorsEXAMPLE LEVEL 2: Follow Corridors

Level 2: Follow CorridorsLevel 2: Follow CorridorsLevel 1: WanderLevel 1: WanderLevel 0 : Avoid CollisionsLevel 0 : Avoid Collisions

Subsumption groups schema-like modules into layers of Subsumption groups schema-like modules into layers of competence, or abstract behaviors.competence, or abstract behaviors.

Higher layers may subsume and inhibit behaviors in lower Higher layers may subsume and inhibit behaviors in lower layers, but behaviors in lower layers are never rewritten or layers, but behaviors in lower layers are never rewritten or replaced.replaced.

The design of layers and component behaviors is difficult. It The design of layers and component behaviors is difficult. It is more of an art than a science. (True for all reactive is more of an art than a science. (True for all reactive architectures)architectures)

Behaviors are released by the presence of stimulus in the Behaviors are released by the presence of stimulus in the environment. (No PLAN component)environment. (No PLAN component)

The releaser is almost always the percept for guiding the The releaser is almost always the percept for guiding the motor schema. motor schema.

Perception is ego-centric and distributed. (sonar plot was Perception is ego-centric and distributed. (sonar plot was relative to the robot, available to any behavior which relative to the robot, available to any behavior which needed it, and the output from perceptual schemas can be needed it, and the output from perceptual schemas can be shared with other layers)shared with other layers)

Subsumption SummarySubsumption Summary

Potential Fields:Potential Fields:Ron ArkinRon Arkin

From http://www.cc.gatech.edu/aimosaic/faculty/arkin

From http://www.cc.gatech.edu/aimosaic/robot-lab/MRLhome.html

Potential Fields PhilosophyPotential Fields Philosophy

The motor schema (what the robot should do) component of a The motor schema (what the robot should do) component of a behavior can be expressed with a potential field’s methodologybehavior can be expressed with a potential field’s methodology• A potential field can be a “primitive” or constructed from A potential field can be a “primitive” or constructed from

primitives which are summed togetherprimitives which are summed together• The output of behaviors are combined using vector summationThe output of behaviors are combined using vector summation

From each behavior, the robot “feels” a vector or forceFrom each behavior, the robot “feels” a vector or force• Magnitude = force, strength of stimulus, or Magnitude = force, strength of stimulus, or velocityvelocity• DirectionDirection

We visualize the “force” as a field, where every point in space We visualize the “force” as a field, where every point in space represents the vector that it would feel if it were at that pointrepresents the vector that it would feel if it were at that point

Potential fields are continuous- Even with a small element at a point can be associated with a vector

How an obstacle will exert a field on the robot and make it run away?

If the robot is close to the obstacle it is inside the potential field and will feel a force that makes it want to face directly away from the obstacle and move away.

If the robot is not within that range then it just sits there because there is no force on it.

Example: Run Away via RepulsionExample: Run Away via Repulsion

5 Primitive Potential Fields5 Primitive Potential Fields

a. Uniform b. Perpendicular c. Attraction d. Repulsion e. Tangential

Common fields in behaviorsCommon fields in behaviors UniformUniform

• Move in a particular direction, corridor followingMove in a particular direction, corridor following RepulsionRepulsion

• Runaway (obstacle avoidance)Runaway (obstacle avoidance) AttractionAttraction

• Move to goalMove to goal PerpendicularPerpendicular

• Corridor followingCorridor following TangentialTangential

• Move through door, docking (in combination with other Move through door, docking (in combination with other fields)fields)

randomrandom• do you think this is a potential field?do you think this is a potential field?

Magnitude Profiles- The length of the arrows gets smaller closer to the object and the way the magnitude of the vectors in the field change is called the magnitude profile.

- Magnitude profiles solve the problem of constant magnitude, which allows the designer to represent reflexivity (response is proportional to the strength of the stimulus).

- The main motivation for magnitude profiles is to fine-tune the behavior.

Linear Drop offIf a robot is far away from the object it will turn and move quickly towards it, then slows up to keep from hitting the object. Mathematically, the rate at which the magnitude of the vectors drops off can be plotted as a straight line.

Exponential Drop offTo have the robot react more the closer it is to the object (constant magnitude). Stronger reaction but more of a taper. The drop off is proportional to the square of the distance for every unit of distance away from the object, the force on the robot drops in half.

Potential Fields and Perceptions Potential Fields are ego-centric because robots perception is ego-centric.

This makes programming easy. The visualization of the entire field may appear to indicate that the robot and the objects are in a fixed, absolute coordinate system but they are not.

The robot computes the effect of the potential field, usually as a straight line, at every update, with no memory of where it was previously or where the robot has moved.

Combining Fields for Emergent BehaviorCombining Fields for Emergent Behavior

obstacleobstacle

goal

If robot were dropped anywhere on this grid,it would want to move to goal and avoid obstacle:

Behavior 1: MOVE2GOAL(attraction field)Behavior 2: RUNAWAY (repulsive field)

The output of each independent behavior is a vector,the 2 vectors are summed to produce emergent behavior

obstacle

goal

Note: In this example, repulsive field only extends for 2 meters;the robot runs away only if obstacle within 2 meters

Note: in this example, robot can sense thegoal from 10 meters away

Fields and Their CombinationFields and Their Combination

Path TakenPath Taken

If robot started at this location, it would take the following pathIf robot started at this location, it would take the following path It would only “feel” the vector for the location, then move It would only “feel” the vector for the location, then move

accordingly, “feel” the next vector, move, etc.accordingly, “feel” the next vector, move, etc. Pfield visualization allows us to see the vectors at all points, but Pfield visualization allows us to see the vectors at all points, but

robot never computes the “field of vectors” just the local vectorrobot never computes the “field of vectors” just the local vector

Robot only feels vectors for this

point when it (if) reaches that point

Update rates, Holonomicity and Local Minima

1. Distance between updates are different (length of the arrows)

Note between t3 and t4, the robot actually goes further without turning and has to turn back as to go to goal. A smoother path would have a faster update.

2. Robots turn in any direction but have to stop and errors develop due to the contact between the wheels and the surface.

3. The vectors only have head and no body, this means the magnitude is 0.0 and that if the robot reaches to that spot it will stop and not move.

Example of using one behavior per sensor

How does the robot see a wall without How does the robot see a wall without reasoning or intermediate representations?reasoning or intermediate representations?

Perceptual schema “connects the dots”, Perceptual schema “connects the dots”, returns relative orientationreturns relative orientation

PS:Find-wall

MS: Perp.

MS: UniformS

Sonars

orientation

Sum all 8 ranges internally

Return the single largest distance and direction

OK, But why isn’t that a representation of OK, But why isn’t that a representation of a wall?a wall?

It’s not really It’s not really reasoning reasoning that it’s a wall, rather it is that it’s a wall, rather it is reacting to the stimulus which happens to be reacting to the stimulus which happens to be smoothed (common in neighboring neurons)smoothed (common in neighboring neurons)

Pfields compared to subsumption

Level 0: RunawayLevel 0: Runaway

Note: multiple instances ofa behavior vs. 1:Could just have 1 Instance of RUN AWAY,Which picks nearest reading;Doesn’t matter, but thisAllows addition of anotherSonar without changing theRUN AWAY behavior

Level 1: WanderLevel 1: Wander

Wander isUniform, but

Changes directionaperiodically

Level 2: Follow CorridorLevel 2: Follow Corridor

Follow-corridor

Example: follow-corridor or follow-Example: follow-corridor or follow-sidewalksidewalk

PerpendicularUniform

Combined

Note use of Magnitude profiles:Perpendicular decreases

PfieldsPfields AdvantagesAdvantages

• Easy to visualizeEasy to visualize• Easy to build up software librariesEasy to build up software libraries• Fields can be parameterizedFields can be parameterized• Combination mechanism is fixed, tweaked with Combination mechanism is fixed, tweaked with

gainsgains

DisadvantagesDisadvantages• Local minima problem (sum to magnitude=0)Local minima problem (sum to magnitude=0)• Jerky motion Jerky motion

Example: Docking BehaviorExample: Docking Behavior

•Arkin and Murphy, 1990, Questa, Grossmann, Sandini, 1995, Tse and Luo, 1998, Vandorpe, Xu, Van Brussel, 1995. Roth, Schilling, 1998, Santos-Victor, Sandini, 1997

Orientation, ratio of pixel counts tangent vectorTotal count attraction vector

Pfields SummaryPfields Summary Reactive Paradigm: SA, sensing is localReactive Paradigm: SA, sensing is local

• Solves the Solves the Open World problemOpen World problem by emulating by emulating biologybiology

• Eliminates the Eliminates the frame problemframe problem by not using any by not using any global or persistent representationglobal or persistent representation

• Perception is direct, ego-centric, and distributedPerception is direct, ego-centric, and distributed Two architectural styles are: Two architectural styles are: subsumption subsumption and and

pfieldspfields Behaviors in pfield methodologies are a tight Behaviors in pfield methodologies are a tight

coupling of sensing to acting; modules are mapped coupling of sensing to acting; modules are mapped to schemas conceptuallyto schemas conceptually

Potential fields and subsumption are logically Potential fields and subsumption are logically equivalent but different implementationsequivalent but different implementations

Pfield problems includePfield problems include• local minima (ways around this)local minima (ways around this)• jerky motionjerky motion• bit of an art bit of an art

Chapter 5Chapter 5

Designing a Reactive Designing a Reactive ImplementationImplementation

ObjectiveObjective

Use schema theory to design and program Use schema theory to design and program behavior using object-oriented behavior using object-oriented programming principle.programming principle.

Design a complete behavioral system, Design a complete behavioral system, including coordinating and controlling including coordinating and controlling multiple concurrent behaviors.multiple concurrent behaviors.

Describe the two methods for assembling Describe the two methods for assembling primitive behaviors into abstract primitive behaviors into abstract behaviors: finite state machines and behaviors: finite state machines and scripts.scripts.

Schema and Behavior ClassesSchema and Behavior Classes

P e cep tu a l S ch e m a M o tor S ch e m a B e h a v io r

C o o rd in a te C o n tro l P ro g a rm

S che m a

P e cep tu a l S ch e m a M o tor S ch e m a B e h a v io r

C o o rd in a te C o n tro l P ro g a rm

B e h a v io r

The reactive movement is characterized by The reactive movement is characterized by

robotics running a very small set of behavior robotics running a very small set of behavior which are combined internally to produce an which are combined internally to produce an overall emergent behavior.overall emergent behavior.

The key components of a reactive The key components of a reactive architecture are the behavior plus the architecture are the behavior plus the mechanism for merging the output of the mechanism for merging the output of the concurrent behaviors. concurrent behaviors.

Series of Behaviors are operating in Series of Behaviors are operating in recognizable sequence such as Pick Up the recognizable sequence such as Pick Up the Trash RobotTrash Robot

Emergent BehaviorEmergent Behavior

Behavior as Object in OOPBehavior as Object in OOP

Schema is well suited for transferring Schema is well suited for transferring theoretical concept in Object-Oriented theoretical concept in Object-Oriented Programming (OOP).Programming (OOP).

Schema theory is used as a bridge between Schema theory is used as a bridge between concept in biological intelligence and robotic concept in biological intelligence and robotic (reactivity and affordance). (reactivity and affordance).

A schema as a programming object would be A schema as a programming object would be a class: Coordinate Control Program, a class: Coordinate Control Program, Behavior, Motor Schema, and Perceptual Behavior, Motor Schema, and Perceptual Schema. Schema.

Behavior as Object in OOPBehavior as Object in OOP

Perceptual Schema takes sensor input and Perceptual Schema takes sensor input and transforms it into data structure called a transforms it into data structure called a percept; percept; it is linked to the sensorit is linked to the sensor..

Motor Schema transform the percept into Motor Schema transform the percept into a vector or other form of representing an a vector or other form of representing an action; it is linked to the robot’s actuators action; it is linked to the robot’s actuators

Behaviors are composed of at least one Behaviors are composed of at least one Perceptual Schema and one Motor Perceptual Schema and one Motor Schema, these schemas act as the Schema, these schemas act as the methods for Behavior classes.methods for Behavior classes.

Primitive and Abstract BehaviorsPrimitive and Abstract Behaviors

Primitive behavior is composed of only one Primitive behavior is composed of only one perceptual schema and one motor schema, perceptual schema and one motor schema, and there is no need to have any coordinated and there is no need to have any coordinated control program. control program.

MonolithicMonolithic Simple mapping Simple mapping Simple method (no multiple methods or objects.Simple method (no multiple methods or objects.

Abstract behavior is composed of multiples Abstract behavior is composed of multiples perceptual schema and motor schema, and perceptual schema and motor schema, and there are farther removed from the sensor there are farther removed from the sensor and actuator than the primitive behavior.and actuator than the primitive behavior.

Primitive and Abstract BehaviorsPrimitive and Abstract Behaviors

M o tor S ch e m a 1 :P e rp e n d icu la r

M o to r S ch e m a 2 :U n ifo rm

P e rce p tua l S ch e m a 1 :F in d -w a lls

V e c to r S u m a tion

F o llo w C o rrid o r

M o to r S ch e m a : fo llo w -w a ll(w h ich -s id e )

B e h a v io r 1 : fo llo w _ w a llle lt

P e rce p tu a l S che m a : fin d -w a ll(p o la r p lo t)

B e h a v io r 2 : fo llo w _ w a llrig h t

V e c to r S u m m a tio n o f B eh a v io rs

F o llo w C o rrid o r

Unmanned Robotics CompetitionUnmanned Robotics Competition Describe the task: specify what the robot Describe the task: specify what the robot

has to do to be successful.has to do to be successful. Describe the robot: determine the basic Describe the robot: determine the basic

physical abilities of the robot and any physical abilities of the robot and any limitation. limitation.

Describe the environment: determine the Describe the environment: determine the situatedness of the robot and identifies situatedness of the robot and identifies perceptual opportunity for the behavior perceptual opportunity for the behavior (Perceptual Schema and its functionality).(Perceptual Schema and its functionality).

Describe how the robot should act in Describe how the robot should act in response to its environment:Identify the set response to its environment:Identify the set of one or more candidate primitive behaviorof one or more candidate primitive behavior

Unmanned Robotics CompetitionUnmanned Robotics Competition

Refine each behavior: concentrate on the Refine each behavior: concentrate on the design of each individual behavior (reactive design of each individual behavior (reactive organization system and activities).organization system and activities).

Test each behavior independently. Test each behavior independently. Test with other behavior: Perform Test with other behavior: Perform

integration testing, where the behaviors integration testing, where the behaviors are combined. This includes testing in are combined. This includes testing in actual environment.actual environment.

Step in designing a reactive Step in designing a reactive behavior systembehavior system

Describe the task

Describe the robot

Describe the environment

Specification& Analysis:ecologicalniche

Implement & refine each behavior

Test each behavior independently

Implementation& unit testing

Test behaviors together SystemTesting

Describe how the robot should act in response to its environment

Design

Finite State Automata (FSA)Finite State Automata (FSA)

Finite State Automata (FSA)Finite State Automata (FSA) A set of popular mechanism for specifying what a A set of popular mechanism for specifying what a

program should be doing at a given time.program should be doing at a given time. Can be represented as a state diagram or a tableCan be represented as a state diagram or a table M = { K, M = { K, ΣΣ, , δδ, s, F } , s, F } M : finite state machine M : finite state machine K : set of all states q K : set of all states q εε K K ΣΣ : set of all inputs : set of all inputs σσ εε ΣΣ δδ : transitions between states : transitions between states s : starting states : starting state F: terminated ( final ) stateF: terminated ( final ) state

Finite State Automata (FSA)Finite State Automata (FSA)

qq σσ δδ

startstart range farrange far follow linefollow line

startstart range nearrange near move aheadmove ahead

follow-linefollow-line range nearrange near move aheadmove ahead

follow-linefollow-line range farrange far follow linefollow line

follow-linefollow-line see finish linesee finish line victory dancevictory dance

move-ahead move-ahead time remainingtime remaining move aheadmove ahead

move-aheadmove-ahead time elapsedtime elapsed follow linefollow line

move-aheadmove-ahead see finish linesee finish line victory dancevictory dance

Finite State automata (FSA)Finite State automata (FSA)

Relationship between FSA and behaviorsRelationship between FSA and behaviors

FSAFSA Behavioral analogBehavioral analog

KK All the behaviors for the taskAll the behaviors for the task

ΣΣ The releasers for each behavior in KThe releasers for each behavior in K

δδ The function that computers the new The function that computers the new statestate

ss The behavior the robot starts in when The behavior the robot starts in when turned onturned on

FF The behavior(s) the robot must reach The behavior(s) the robot must reach to terminateto terminate

Finite State automata (FSA)Finite State automata (FSA)Pick Up the Trash FSAPick Up the Trash FSA

Finite State automata (FSA)Finite State automata (FSA)Pick Up the Trash FSAPick Up the Trash FSA

Finite State automata (FSA)Finite State automata (FSA) Pick Up the Trash FSAPick Up the Trash FSA

Finite State automata (FSA)Finite State automata (FSA)

AdvantagesAdvantages• Formal mechanismFormal mechanism• Have to fill in the table, so a way of spotting Have to fill in the table, so a way of spotting

unforeseen effectsunforeseen effects• Can be implemented in a number of waysCan be implemented in a number of ways

DisadvantagesDisadvantages• Hard to keep up with implicit behaviorsHard to keep up with implicit behaviors

Ex. Avoid obstacle is often running throughout ALL Ex. Avoid obstacle is often running throughout ALL states, gets tedious to express it formallystates, gets tedious to express it formally

• Tend to add explicit variable for transitions, Tend to add explicit variable for transitions, rather than rely on emergent behavior from rather than rely on emergent behavior from environmentenvironment

Abstract BehaviorsAbstract Behaviors

Template abstract behaviors Template abstract behaviors - pick-up-trash (trash-color, trash can color, size-trash pick-up-trash (trash-color, trash can color, size-trash

can)can)- Different initial conditionsDifferent initial conditions

ImprintingImprinting- Detect best value for sensing. ( red can )Detect best value for sensing. ( red can )

ExceptionsExceptions- Substitute behavior under certain contentions Substitute behavior under certain contentions

(different values of red, raster scan)(different values of red, raster scan)

ScriptsScripts Originally used in natural language processing.Originally used in natural language processing. Think of the robot and the tasks in terms of a screenplay.Think of the robot and the tasks in terms of a screenplay. Causal chain.Causal chain.

ScriptScript Behavior AnalogBehavior Analog ExamplesExamples

GoalGoal TaskTask Find victims in rubbleFind victims in rubble

PlacesPlaces Environment, Environment, applicability or applicability or “taskability” for new “taskability” for new taskstasks

Collapsed buildingsCollapsed buildings

ActorsActors BehaviorsBehaviors explore(), dance(), avoid(), explore(), dance(), avoid(), crawl, move2void, crawl, move2void, move2victimmove2victim

Props, cuesProps, cues PerceptsPercepts Voids: Dark, concaveVoids: Dark, concave

Victims: heat, motion, colorVictims: heat, motion, color

Causal ChainCausal Chain Sequence of Sequence of behaviorbehavior

Explore, dance, move2void, Explore, dance, move2void, crawl, move2victim, crawl, move2victim, dropRadiodropRadio

SubscriptsSubscripts Exception handlingException handling If lose communications, If lose communications, return homereturn home

ScriptsScriptsPick up the trash example:Pick up the trash example:

for each update…for each update…\\ look for props and cues first: cans, trash cans, gripper\\ look for props and cues first: cans, trash cans, gripperrstatus=extract_color(red, rcx, rSize);rstatus=extract_color(red, rcx, rSize);if(rStatus==TRUE)if(rStatus==TRUE) SEE_RED=TRUE;SEE_RED=TRUE;elseelse SEE_RED=FALSE;SEE_RED=FALSE;bStatus=extract_color(blue,bcx,bSize);bStatus=extract_color(blue,bcx,bSize);if(bStatus==TRUE){if(bStatus==TRUE){ SEE_BLUE=TRUE; NO_BLUE=FALSE;SEE_BLUE=TRUE; NO_BLUE=FALSE;} else {} else { SEE_BLUE=FALSe; NO_BLUE=TRUE;SEE_BLUE=FALSe; NO_BLUE=TRUE;}}AT_BLUE=looming(size,bSize);AT_BLUE=looming(size,bSize);gStatus=gripper_status();gStatus=gripper_status();if(gStatus==TRUE) {if(gStatus==TRUE) { FULL=TRUE; EMPTY=FALSE;FULL=TRUE; EMPTY=FALSE;} else {} else { FULL=FALSE; EMPTY=TRUE;FULL=FALSE; EMPTY=TRUE;}}

Pick up the trash example:Pick up the trash example:

for each update…for each update…\\ look for props and cues first: cans, trash cans, gripper\\ look for props and cues first: cans, trash cans, gripperrstatus=extract_color(red, rcx, rSize);rstatus=extract_color(red, rcx, rSize);if(rStatus==TRUE)if(rStatus==TRUE) SEE_RED=TRUE;SEE_RED=TRUE;elseelse SEE_RED=FALSE;SEE_RED=FALSE;bStatus=extract_color(blue,bcx,bSize);bStatus=extract_color(blue,bcx,bSize);if(bStatus==TRUE){if(bStatus==TRUE){ SEE_BLUE=TRUE; NO_BLUE=FALSE;SEE_BLUE=TRUE; NO_BLUE=FALSE;} else {} else { SEE_BLUE=FALSe; NO_BLUE=TRUE;SEE_BLUE=FALSe; NO_BLUE=TRUE;}}AT_BLUE=looming(size,bSize);AT_BLUE=looming(size,bSize);gStatus=gripper_status();gStatus=gripper_status();if(gStatus==TRUE) {if(gStatus==TRUE) { FULL=TRUE; EMPTY=FALSE;FULL=TRUE; EMPTY=FALSE;} else {} else { FULL=FALSE; EMPTY=TRUE;FULL=FALSE; EMPTY=TRUE;}}

\\ index into the correct step in the \\ index into the correct step in the causal chaincausal chain

if(EMPTY) {if(EMPTY) { if(SEE_RED) {if(SEE_RED) { move_to_goal(red);move_to_goal(red); } else {} else { wander();wander(); }}} else {} else { grab_trash();grab_trash(); if(NO_BLUE){if(NO_BLUE){ wander();wander(); } else if (AT_BLUE){} else if (AT_BLUE){ drop_trash();drop_trash(); } else if (SEE_BLUE) {} else if (SEE_BLUE) { move_to_goal(blue);move_to_goal(blue); }}}}

\\ index into the correct step in the \\ index into the correct step in the causal chaincausal chain

if(EMPTY) {if(EMPTY) { if(SEE_RED) {if(SEE_RED) { move_to_goal(red);move_to_goal(red); } else {} else { wander();wander(); }}} else {} else { grab_trash();grab_trash(); if(NO_BLUE){if(NO_BLUE){ wander();wander(); } else if (AT_BLUE){} else if (AT_BLUE){ drop_trash();drop_trash(); } else if (SEE_BLUE) {} else if (SEE_BLUE) { move_to_goal(blue);move_to_goal(blue); }}}}

ScriptsScripts

AdvantagesAdvantages• A more storyboard like way of thinking about A more storyboard like way of thinking about

the behaviorsthe behaviors• If-then, switch style of programming like FSAIf-then, switch style of programming like FSA• Since a Script is “in” a behavior, other Since a Script is “in” a behavior, other

behaviors such as avoid can be running behaviors such as avoid can be running concurrently without having to appear “in” the concurrently without having to appear “in” the scriptscript

• Exception handling is a big plus in Real LifeException handling is a big plus in Real Life DisadvantagesDisadvantages

• Can be a bit of overkill for simple sequences, Can be a bit of overkill for simple sequences, especially with C++especially with C++