Reactive & Behavior-Based Paradigm

Objectives To understand two dominant methods for combining behaviors in a

reactive architecture: Subsumption Motor schemas (potential fields)

Evaluate subsumption and potential fields architectures in terms of: Support for modularity Niche targetability Ease of portability to other domains Robustness

Be able to program using potential fields

Be able to construct a new potential field from primitive potential fields, and sum potential fields to generate an emergent behavior

Reactive Systems

A reactive robotic system tightly couples perception to action without the use of intervening abstract representations or time history The tight coupling of sensing and acting allows robots to

operate in real-time

Recall: reactive paradigm

Attributes of Reactive Paradigm

All actions are accomplished through behaviors Behaviors are a direct mapping of sensory inputs to a pattern of

motor actions that are used to achieve a task We treat it as perceptual schema + motor schema

No PLAN component, all robotic activities emerge as the behaviors operating either in sequence or concurrently S-A doesn’t specify how the behaviors are coordinated; it is

addressed by architecture

Sensing is local to each behavior, or behavior specific One sensor, one behavior Multiple sensors, one perceptual schema, one behavior (sensor

fusion)

Characteristics of Reactive Paradigm

Robots are situated agents operating in an ecological niche It is an integrated part of the world

Behaviors serve as basic building blocks for robotic actions Robotic activities emerge as the result of these behaviors

operating in sequence or concurrently

Use of explicit abstract representational knowledge is avoidedin the generation of a response Only local, behavior-specific sensing is allowed

Animal models of behavior often serve as a basis for these systems

These systems are inherently modular from a software design perspective

Comparing Alternative Architectures

A reactive architecture provides mechanism for: triggering behaviors determining what happens when multiple behaviors are active in the

same time

Commonalities: Emphasis on the importance of coupling sensing and action tightly Avoidance of representational symbolic knowledge Decomposition into contextually meaningful units (behaviors)

Distinctions: Granularity of behavioral decomposition Basis for behavior specification (ethological, situated activity,

experimental) Response encoding method (discrete vs. continuous) Coordination methods (competitive vs. cooperative)

Example Reactive/Behavior-Based Robots

Genghis, MIT

Subsumption

Callisto, GATech

Potential Field

Evaluating Architectures

How do we evaluate an architecture’s suitability for a particular problem? Hardware targetability Support for modularity Robustness Timeliness in development Run time flexibility Performance effectiveness

Subsumption Architecture Developed in mid-1980s by Rodney Brooks, MIT

Tenets of the Subsumption Architecture

Complex behavior need not be the product of a complex control system

Intelligence is the eye of the observer The world is its own best model Simplicity is a virtue Robots should be cheap Robustness in the presence of noisy or failing sensors is a

design goal Planning is just a way of avoiding figuring out what to do

next All onboard computation is important Systems should be built incrementally No representation. No calibration. No complex computers. No

high-bandwidth communication

Categorization of Subsumption Architecture

Subsumption Robots Allen Tom and Jerry Genghis and Attila Squirt Toto Seymour Tito Polly Cog

“Veteran” robots of the MIT AI Lab using the subsumption architecture

Some Aspects of Subsumption A behavior is a network of sensing and acting modules, which are

augmented finite state machines (AFSM)

Modules are grouped into layers of competence The layers reflect a hierarchy of intelligence Lower layers: basic survival functions Higher layers: more goal-directed actions

Modules in a higher layer can override, or subsume, the output from behaviors in the next lower layer Behavioral layers operate concurrently and independently

The use of internal state is avoided Some internal state is needed for releasing behaviors, but good behavioral

designs minimize this

A task is accomplished by activating the appropriate layer, which then activates the lower layers below it, and so on However, subsumption style systems are not easily taskable

Coordination in Subsumption

“Subsumption” comes from coordination process used between layered behaviors of architecture

Complex actions subsume simpler behaviors Fixed priority hierarchy defines topology Lower levels of architecture have no “awareness”

of upper levels

Coordination has two mechanisms: Inhibition Suppression

Inhibition and Suppression

Inhibition: the output of the subsuming module is connected to the

output of another module if the subsuming module is “on” or has any value, the

output of the subsumed module is blocked or turned “off” acts like a faucet

Suppression the output of the subsuming module is connected to the

input of another module if the output of the subsuming module is on, it replaces

the normal input to the subsumed module acts like a switch

Foraging Example

Foraging: Robot moves away from home base looking for

attractor objects When detect attractor object, move toward it,

pick it up, and return it to home base Repeat until all attractors been collected

High-level behaviors: Wander Acquire (pick up) Retrieve (homing)

Organization of Subsumption-Based Foraging Robot

I

I

I

Example

Develop a robot that can follow the center of the corridor, while avoiding obstacles

This robot has multiple sonars (or other range sensors), each pointing in a different direction; and two actuators, one for driving forward, one for turning

Example Perception: Polar Plot

if sensing is ego-centric, canoften eliminate need for memory, representation

Plot is ego-centric

Plot is distributed (available to whoever wants to use it) Although it is a representation in the sense of being a data

structure, there is no memory (contains latest information) and no reasoning (2-3 means a “wall”)

Level 0: Runawayrunaway 0

wander 1

follow-corridor 2

HALT

COLLIDE

PS MS

RUN AWAYPS MS

Level 1: Wanderrunaway 0

wander 1

follow-corridor 2

encoders

AVOID

PS

MS

WANDER

PS MS

Note sharing ofPerception, fusion

What wouldthis do?

Level 2: Follow Corridorsrunaway 0

wander 1

follow-corridor 2

STAY-IN-MIDDLE

PS MS

Class Exerciserunaway 0

wander 1

move2light 2

LIGHT move-to-light

S

Genghis Subsumption Design

Behavioral layers implemented: Standup Simple walk Force balancing Leg lifting Whiskers Prowling Steered prowling

Two motors per leg:advance, which swings leg back and forth

balance, which lifts leg up and down

http://www.youtube.com/watch?v=BUxFfv9JimUhttp://www.youtube.com/watch?v=RKeBl0-msGQ

Genghis AFSM Network

Evaluation of Subsumption Strengths:

Hardware retargetability: subsumption can compile down directly onto programmable-array logic circuitry

Support for parallelism: each behavioral layer can run independently and asynchronously

Niche targetability: custom behaviors can be created for specific task-environment pairs

Null (not strength/not weakness): Robustness: can be successfully engineered into system but is often

hard-wired and hard to implement Timeliness for development: some support tools exist, but significant

learning curve exists

Weakness: Run time flexibility: priority-based coordination mechanism, ad hoc

aspect of behavior generation, and hard-wired aspects limit adaptation of system

Support for modularity: behavioral reuse is not widely done in practice

Next …

Motor schemas

Potential fields

Motor Schemas

Based upon schema theory: Explains motor behavior in terms of concurrent

control of many different activities Schema stores both how to react and the way

that reaction can be realized A distributed model of computation

Motor Schemas Developed by Arkin in 1980s Based on biology’s schema theory Behavioral responses are all represented as vectors generated using

a potential fields approach Coordination is achieved by vector addition

Categorization of Motor Schemas

Differences of Motor Schemas v.s. Other Behavioral Approaches

Behavioral responses are all represented as vectors generated using a potential fields approach

Coordination is achieved by vector addition

No predefined hierarchy exists for coordination; instead, behaviors are configured at run-time

Pure arbitration is not used; each behavior can contribute in varying degrees to robot’s overall response

Perception-Action Schema Relationships

Schema-Based Robots

HARV George Ren and Stimpy Buzz Io, Callisto, Ganymede Mobile manipulator

Output of Motor Schemas Defined as Vectors

Output Vector: consists of both orientation and magnitude components

Vmagnitude denotes magnitude of resultant response vector Vdirection denotes orientation Vector represents a force Typically drawn as an arrow

Potential field: array (or field) of vectors representing space

Length of arrow = m = magnitude

Angle of arrow = d = direction

Potential Fields (con’t.) Vector space is 2D world, like bird’s eye view of map Map divided into squares, creating (x, y) grid Each element represents square of space Perceivable objects in world exert a force field on surrounding space

Five Primitive Potential Fields

Magnitude Profiles

Change in velocity in different parts of the field

Field closer to an attractor/repellor will be stronger

Programming a Single Potential Field

Repulsive field with linear drop-off:

Pseudocode for Single Repulsive Field

Runaway Behavior

Assume we have a robot with a single sonar readSonar() is a perceptual schema that returns

a distance turn(dir) will turn the robot in the “dir” direction forward(mag) will set the speed of the robot

How do we use the repulsive potential field to program the runaway behavior?

vector runaway() {double reading;reading = readSonar(); // psvoutput = repulsive(reading, MAX_DIST); // msreturn voutput;

}while (robot is on) {

vrunaway = runaway();turn(vrunaway.direction);forward(vrunaway.magnitude);

}

Important Note

Entire field does not have to be computed

Only portion of field affecting robot is computed

Robot uses functions defining potential fields as its position to calculate component vector

Fine-tune update rate to generate a smooth path

More Motor Schema Encodings Move-to-goal:

Vmagnitude = fixed gain value Vdirection = towards perceived goal Tolerance around the goal +/- 0.5m

Avoid-static-obstacle:

More Motor Schema Encodings Stay-on-path:

More Motor Schema Encodings

Move-ahead:

Noise:

Combing Fields/Behaviors Compute each behavior’s potential field Sum vectors at robot’s position to get resultant

output vector

?

+

=

Issues: local minima: vectors may

sum to zero

R = Σ (Gi · Ri)

Solutions?

Solutions for Dealing with Local Minima

Inject noise, randomness: “Bumps” robot out of minima

Include “avoid-past” behavior: Remember where robot has been and attracts the robot to

other places

Use “Navigation Templates”: The “avoid” behavior receives as input the vector summed

from other behaviors Gives “avoid” behavior a preferred direction

Insert tangential fields around obstacles

Recall Our Previous Example

Develop a robot that can follow the center of the corridor, while avoiding obstacles

This robot has multiple sonars (or other range sensors), each pointing in a different direction; and two actuators, one for driving forward, one for turning

Sonsor

0

12

3

4

5 6

7

Potential Field Solution (1)

Runaway behavior

while (robot is on) {vector.mag = vector.dir = 0; // initializationfor (i = 0; i <= numSonar; i++) {

vectorCurrent = Runaway(i);vectorOutput = VectorSum(Vector, vectorCurrent);

}turn(vector.dir);forward(vector.mag);

}

runaway 0

wander 1

follow-corridor 2

Potential Field Solution (2)

Collide is treated as “panic” situations, triggering an emergency response outside the potential field framework

runaway 0

wander 1

follow-corridor 2

Robot in a Box Canyon

0

12

3

4

5 6

7

What does the vector field look like?Can this robot escape this canyon?

Alternative Implementation of Runaway

Runaway to process all 8 sonar readings

Can be used only with a robot that has 8 range sensors

Potential Field Solution (3)

Wander behavior Follow corridor

runaway 0

wander 1

follow-corridor 2

Motor Schemas Achieve Behavioral Fusion via Vector Summation

S-R Diagram for Cooperating Behaviors in Foraging

Wander

Avoid-obstacle Σ

Σ

Deliver-home

Σ

no attractor

obstacle

robot

attractor

obstacle

robot

local minima

homebase

obstacle

robot

Sequencer

Avoid-obstacle

Avoid-obstacle

Avoid-obstacle

Avoid-obstacle

Avoid-obstacle

Noise

Noise

Goto-attractor

Wander

Acquire

Deliver

local minima

Sequencing of Motor Schemas

Can sequence motor schemas if one activity needs to be completed before another

Callisto, the Foraging Robot @Georgia Tech Winners in a robot competition at AAAI-94 http://www.cc.gatech.edu/ai/robot-lab/research/aaai94.html

Trash-collecting robots

More Exercises

Construct a potential field for the following cases: move to a goal position with an obstacle in between move through a door

Please identify the potential fields that you use and draw the potential field

Advantages and Disadvantages of the Potential Field Approach

Advantages: It is easier to visualize the robot’s overall

behavior It is easier to combine fields It can be parameterized It can be extended from 2D to 3D

Disadvantages: Local minima problem

Design in Motor-Schema-Based System

Characterize problem in terms of motor behaviors needed to accomplish task

Decompose motor behaviors to most primitive level, using biological studies when feasible

Develop formulas to express robot’s reaction to perceived environmental events

Conduct simple simulation studies to assess desired behaviors’ approximate performance

Determine perceptual requirements needed to satisfy motor schema inputs

Design specific perceptual algorithms Integrate control system onto robot Test and Evaluate Iterate and expand as needed

Evaluation of Reactive Architectures

Support for modularity Both decompose the actions and perceptions into

behaviors

Niche targetability Both high The use of direct perception emphasizes that reactive

robots are constructed to fill a niche

Portability to other domains Limited to applications that can be accomplished with

reflexive behaviors Potential fields approach performs better

Robustness No specific mechanisms to support

Class So Far

History of robotics Robot software design strategies Robotics paradigms and architectures Behavior encoding, combinations, etc. Pros, cons, evaluation of various architectures Programming implementations/pseudocodes of

above