Adaptation in Embodied & Situated Agents

Adaptation in embodied & situated agents

Author: Claudio MartellaCollaborators: Dott. Stefano Nolfi (ISTC - CNR)

Prof. N.A. Borghese (AIS Lab - UniMi)

October, 2011

1

Tuesday, October 11, 11

• the behavior might be too complex for the designer to control

• the environment is noisy and not perfect

• the world is unpredictable

It is difficult to build autonomous systems through a top-down approach:

2

The problem


Evolutionary robotics is a branch of robotics that uses evolutionary methodologies

to develop controllers for autonomous robots.

Nolfi, Floreano [2004] - MIT Press

3


The objective

We wanted to analyze the possibility of applying adaptive processes to embodied & situated agents

considering evolutionary, individual and social learning.

4


E&S agents

• Embodied: the agent can exploit the characteristics of the robot (shape, sensors, actuators etc.).

• Situated: the solution can exploit the possible interactions that the environments offers.

5


The methodologyE-puck Robot Simulation

Problem: categorize 10 objects (Good, Poisonous)6


The evolutionary process

7


1st goal

8

Implement an algorithm for individual learning.

The algorithm should start with one set of candidate parameters

and it would modify them by trial & error.

Decision: start from Simulated Annealing *

* "Optimization by Simulated Annealing", Kirkpatrick, S.; Gelatt, C. D.; Vecchi, M. P. (1983) - Science


Simulated Annealing

9

Temperature:

It probabilistically accepts mutations that decrease

the fitness.

The probability decreases with time.

It allows the algorithm to jump out of local minima.


Stochasticity in E&SEvaluation depends on

the (random) initial conditions:

10


The intuition

0

0.225

0.45

0.675

0.9

100 200 300 400 500

Temperature

0

0.225

0.45

0.675

0.9

10 20 30 40 50

Stochasticity

Probability of accepting negative mutations decreases with the

increase of time

Probability of accepting negative mutations decreases with the

increase of #evaluations11


Contributions

• Simplifies the algorithm

• Better performance (~10% improvement)

• Lighter algorithm (~50% less evaluations for us)

• Remove Temperature

• Start with few evaluations and increase with time

Results

12

Substitute external stochasticity with internal:


2nd goal

13

Implement an algorithm for social learning.

The algorithm should take advantage of the interaction with an expert agent

to acquire an adaptive solutionthat is improved and/or in less time.

Decision: apply individual learning to imitation.


Why?

14

Social learning should avoid reinventing the wheel.

In principle, when guided, learning is faster & safer.

It should be the basis for cultural evolution.


How?

15

There are simpler forms of social learning:

• social facilitation

• contagious behavior

• stimulus enhancement


How (technically)?

16

Fitness function: student should learn to give outputs similar to the agent’s, given the same input.


How (technically)?

17

fit = fitsoc

· (1� ↵) + fitind

· ↵

↵ = cN

Pure imitation brings to under-fitting individuals.We introduced a hybrid approach.


Contributions

• Performance on the problem is not improved

• Adaptive behavior is acquired faster

• More agents acquire an adaptive behavior

• Modeled social learning with simple form of imitation

• Modeled hybrid social-individual learning approach

Results

18


Intuitive interpretation

19

Social learning as a method for promising initial parameters selection.

Social learning as a method for jumping out of local maxima.

parameters space solutions space


Questions?

20


Technology

Adaptation in Embodied & Situated Agents