December 1999 1 Recent Research Biologically-inspired Visual Landmark Navigation for Mobile Robots Collaborative Work with Bianco

QuickTime™ and aPhoto - JPEG decompressorare needed to see this picture.December 1999 1

Recent Research

Biologically-inspired Visual Landmark Navigation for Mobile Robots

Collaborative Work with Bianco


Mobile Robot Navigation

Robot Navigation relies answering the questions:

–Where am I?

–Where are other places relative to me?

–How do I get to other places from here?

Possible answers:

–Classical robotic techniques

–New trend: biologically-inspired methods


Biological-inspiration

Animals and insects are proficient in visual navigation (Papi 1992)

The use of natural visual landmarks by insects for navigation have been well documented (Wehner 1992)

Strategies for the selection of natural landmarks by insects has been reported (Lehrer 1993, Zeil 1993)

Many models have been introduced without formal methods (Trullier et al. 1997)


Related Issues

Navigation can be considered as a four-level hierarchy

–guidance

–place recognition-triggered response

–topological navigation

–metric navigation

We perform guidance: the agent is guided by a spatial distribution of landmarks

–we do not use maps

–we do not know our position in reference co-ordinates


Acquiring Visual Landmarks

“A landmark must be reliable and landmarks which appear to be appropriate for human beings are not necessarily appropriate for robots because of the different sensor and matching apparatus.”

Matàric 1990, Thrun 1996

If we can establish what is meant by reliability for given sensors and matching schemes then the problem of landmark selection is automatically solved!

Reliability depends on sensor and matching scheme

– Sony NTSC camera and Fujitsu Tracking card (TRV) Landmarks are based on the image correlation concept.


Definition of a Landmark

A landmark is a region within a whole image

– TRV performs 16 x 16 SAD correlation

– A correlation matrix is generated

(ox,oy)

(ox-8*mx,oy -8*my)

(ox+7*mx,oy +7*my)

16*mx

16*my

Template

Frame


0

5

10

15

0

5

10

15

0

5

x 104

0

5

10

15

0

5

10

15

0

5000

Uniqueness of Landmarks

Different landmarks have different correlation matrices


0

5

10

15 0

5

10

15

0

5

x 104

Reliability of Landmarks

We define the reliability of a landmark as

g

gr

′−=1 where g’ is a local minimum

found in the neighbourhood of g, the global minimum.

Maximising r, coupled with different template sizes, we can select unique landmarks.

Selecting Landmarks

QuickTime™ and aPhoto - JPEG decompressorare needed to see this picture.December 1999

Magnification size = 3 & 4 Magnification size = 5 & 6

9

How “dynamically” reliable are our landmarks?

Through a phase directly inspired by wasps and bees, the robustness of statically chosen landmarks is tested.

Turn Back and Look

Titolo:e:\winnt\profiles\corrado\desktop\freccine2.epsAutore:Micrografx Graphics EngineAnteprima:L'immagine EPS non è stata salvata con l'anteprima inclusa in essa.Commento:L'immagine EPS potrà essere stampata con una stampante PostScript e non con altri tipi di stampante.


The robot moves with stereotyped movements The camera continuously points toward the goal

10

Two frames from a typical TBL phase

Turn Back and Look


Numbers show the reliability factors of the landmarks

11

The reliability r is constantly monitored for each landmark during TBL.

Only those landmarks whose r is above a threshold e are considered.

Small perturbations (light, position, etc.) are produced with TBL and this represents a framework for testing the reliability of real navigation tasks.

“Only strong individuals can survive through a selection phase” (Murray Gell-Mann, The Quark and the Jaguar, 1994)

Turn Back and Look


TBL produces small perturbations: light, perspective, size...

Perturbations

Titolo:c:\temp\tbl-lm.epsAutore:MATLAB, The Mathworks, Inc.Anteprima:L'immagine EPS non è stata salvata con l'anteprima inclusa in essa.Commento:L'immagine EPS potrà essere stampata con una stampante PostScript e non con altri tipi di stampante.


The underlying principle is based on the model proposed by Cartwright and Collet to explain bee behaviour.

It mimics the behaviour of a bee quite well BUT a 2D extension is required

Key Point for the extension:

– A landmark is attracted toward its original position and size

Landmark Navigation


Displacement from the original position and size is suitable for extracting navigation information from landmarks.

Landmark Navigation

Original position and size New position and sizeQuickTime™ and aPhoto - JPEG decompressorare needed to see this picture.

December 1999 15

Let be the difference between the original and present positions of the landmarks.

Let be the weight for size difference of the landmarks.

The landmark attraction vector is given by:

Landmark Navigation

ldr

lW

lW =

Mxy

mxy

if Mxy

mxy

>1

−mxy

Mxy

otherwise

⎧

⎨ ⎪

⎩ ⎪

r v l =

r d l ⋅Wl


l

r v

16

By fusing all the landmark attraction vectors

through weighted averaging:

we obtain the final

navigation vector

Landmark navigation

r V = Vx Vy[ ]=

r v l ⋅s(rl )

l=1

L

∑

s(rl)l=1

L

∑


Typical image input frame

r V

17

Images from a Navigation Experiment


A navigation vector field: for each (x,y) can be computed

Landmark navigation

2165

6

5234

2294

3742

238

5623

4899

1542

2774

95

193

207

134

405

17

196

72

1340

121

192

192

3362

909

5358

4431

4551

1076

4032

3450

1017

5733

1317

2 4717

5023

5702

655

1025

1155

6824

1677

7

5596

1686

31532

3

1400

1

1452

612

444

3503

7588

5228

8002

5210

9118

981

2263

2047

2240

2341

3768

1271

7

1853

4243

8675

5208

5934

6891

1225

1351

2

1185

7

7510

3006

2134

1307

2413

2200

1

1020

cm 60

cm

60 cm

GOAL POSITION

720 cm


There is evidence of a potential field when biologically-based navigation is considered (Voss 1995, Gaussier 1998)

In this case, a potential function U(x,y) such that

can drive the movements of the robot. A necessary and sufficient condition for U to exist is that the vector

field is conservative, that is:

or, alternatively

Visual Potential Field

r V = Vx Vy[ ]=

∂U∂x

∂U∂y

⎡

⎣ ⎢ ⎤

⎦ ⎥

Vr

∂Vx

∂y =

∂Vy

∂x

r V od

r r =0

c∫


Partial derivatives

and

are computed numerically.

Computation of Partial Derivatives

∂Vx

∂y ∂Vy

∂x


TBL affects the conservativeness according to different thresholds Plotting for different threshold values of e yields:

TBL affects Conservativeness

∂Vx

∂y −

∂Vy

∂x


e=0 e=0.1

22

As e -->1 the vector field becomes conservative and computation of the potential field can be possible

TBL affects Conservativeness


e=0.2 e=2.5

23

Different Potential Fields U can be generated from values of e.

Computation of the Potential Field


e=0 e=0.1

24

As e -->1 the potential field is suitable to drive navigation

Computation of the Potential Field


e=0.1 e=0.25

25

When a smaller template size is considered, the potential field basin has a different shape:

– deeper at the goal position.

– a reduced basin of attraction. Example size 4, e=0.2

Visual potential field

Titolo:c:\temp\u4.epsAutore:MATLAB, The Mathworks, Inc.Anteprima:L'immagine EPS non è stata salvata con l'anteprima inclusa in essa.Commento:L'immagine EPS potrà essere stampata con una stampante PostScript e non con altri tipi di stampante.


• Basic navigation rule:

if then continue using the last navigation vector

else give the robot the currently computed navigation vector

Equipment– Nomad200

– Sony EVI-D30

– Fujitsu Colour TRV

Experimentation

Var(r V ) ≥σ


The Environment

102

0 cm

CUPBOARD(h=210cm)

TABLE (h=70cm)

60

cm

60 cm

GOAL POSITION

COLUMN

720 cm


Size = 6 Threshold e = 0

Experiment A

1020

cm

CUPBOARD(h=210cm)

TABLE (h=70cm)

60

cm

60 cm

GOAL POSITION

COLUMN

720 cmWALL

1

G1

2

3

4

G4 5

G5

6

7

8

G8

9G9

10G10

11

G11


Size = 6 Threshold e = 0.2

Experiment B

1020

cm

CUPBOARD(h=210cm)

TABLE (h=70cm)

60

cm

60 cm

GOAL POSITION

COLUMN

720 cmWALL

1

G1

2

G2

3

4G4

5

G5

6

7

8G8

9

G9

10

G10

11

G11 G6

G7


Experiment C

1020

cm

CUPBOARD(h=210cm)

TABLE (h=70cm)

60

cm

60 cm

GOAL POSITION

COLUMN

720 cmWALL

1

G1

2

G2

3

4

G4

5

G5

6

7

8G8

9

G9

10

G10

11

G6

G7

Size = 5 Threshold e = 0.2


Major results– Self-selection of natural landmarks

– Theory of visual potential

– Landmark definition based on reliability

– Landmark navigation can been formalised as driven by a potential field

– Invariants or Transformations are not needed

Most importantly– TBL affects the conservativeness of the vector field

– strong landmarks = conservativeness = potential field

– Biologically-inspired navigation methods are effective

Conclusions


Documents

December 1999 1 Recent Research Biologically-inspired Visual Landmark Navigation for Mobile Robots Collaborative Work with Bianco