View
2.941
Download
1
Category
Preview:
DESCRIPTION
Citation preview
Adaptation in embodied & situated agents
Author: Claudio MartellaCollaborators: Dott. Stefano Nolfi (ISTC - CNR)
Prof. N.A. Borghese (AIS Lab - UniMi)
October, 2011
1
Tuesday, October 11, 11
• the behavior might be too complex for the designer to control
• the environment is noisy and not perfect
• the world is unpredictable
It is difficult to build autonomous systems through a top-down approach:
2
The problem
Tuesday, October 11, 11
Evolutionary robotics is a branch of robotics that uses evolutionary methodologies
to develop controllers for autonomous robots.
Nolfi, Floreano [2004] - MIT Press
3
Tuesday, October 11, 11
The objective
We wanted to analyze the possibility of applying adaptive processes to embodied & situated agents
considering evolutionary, individual and social learning.
4
Tuesday, October 11, 11
E&S agents
• Embodied: the agent can exploit the characteristics of the robot (shape, sensors, actuators etc.).
• Situated: the solution can exploit the possible interactions that the environments offers.
5
Tuesday, October 11, 11
The methodologyE-puck Robot Simulation
Problem: categorize 10 objects (Good, Poisonous)6
Tuesday, October 11, 11
The evolutionary process
7
Tuesday, October 11, 11
1st goal
8
Implement an algorithm for individual learning.
The algorithm should start with one set of candidate parameters
and it would modify them by trial & error.
Decision: start from Simulated Annealing *
* "Optimization by Simulated Annealing", Kirkpatrick, S.; Gelatt, C. D.; Vecchi, M. P. (1983) - Science
Tuesday, October 11, 11
Simulated Annealing
9
Temperature:
It probabilistically accepts mutations that decrease
the fitness.
The probability decreases with time.
It allows the algorithm to jump out of local minima.
Tuesday, October 11, 11
Stochasticity in E&SEvaluation depends on
the (random) initial conditions:
10
Tuesday, October 11, 11
The intuition
0
0.225
0.45
0.675
0.9
100 200 300 400 500
Temperature
0
0.225
0.45
0.675
0.9
10 20 30 40 50
Stochasticity
Probability of accepting negative mutations decreases with the
increase of time
Probability of accepting negative mutations decreases with the
increase of #evaluations11
Tuesday, October 11, 11
Contributions
• Simplifies the algorithm
• Better performance (~10% improvement)
• Lighter algorithm (~50% less evaluations for us)
• Remove Temperature
• Start with few evaluations and increase with time
Results
12
Substitute external stochasticity with internal:
Tuesday, October 11, 11
2nd goal
13
Implement an algorithm for social learning.
The algorithm should take advantage of the interaction with an expert agent
to acquire an adaptive solutionthat is improved and/or in less time.
Decision: apply individual learning to imitation.
Tuesday, October 11, 11
Why?
14
Social learning should avoid reinventing the wheel.
In principle, when guided, learning is faster & safer.
It should be the basis for cultural evolution.
Tuesday, October 11, 11
How?
15
There are simpler forms of social learning:
• social facilitation
• contagious behavior
• stimulus enhancement
Tuesday, October 11, 11
How (technically)?
16
Fitness function: student should learn to give outputs similar to the agent’s, given the same input.
Tuesday, October 11, 11
How (technically)?
17
fit = fitsoc
· (1� ↵) + fitind
· ↵
↵ = cN
Pure imitation brings to under-fitting individuals.We introduced a hybrid approach.
Tuesday, October 11, 11
Contributions
• Performance on the problem is not improved
• Adaptive behavior is acquired faster
• More agents acquire an adaptive behavior
• Modeled social learning with simple form of imitation
• Modeled hybrid social-individual learning approach
Results
18
Tuesday, October 11, 11
Intuitive interpretation
19
Social learning as a method for promising initial parameters selection.
Social learning as a method for jumping out of local maxima.
parameters space solutions space
Tuesday, October 11, 11
Questions?
20
Tuesday, October 11, 11
Recommended