View
31
Download
1
Category
Tags:
Preview:
DESCRIPTION
Insect neural networks as a visual collision detection mechanism in automotive situations. Richard Stafford (1), Matthias S. Keil (2), Shigang Yue (1), Jorge Cuadri-Carvajo (2), F. Claire Rind (1) 1) School of Biology, Ridley Building, University of Newcastle upon Tyne, NE1 7RU, UK. - PowerPoint PPT Presentation
Citation preview
Insect neural networks as a visual collision detection mechanism in automotive situations
Richard Stafford (1), Matthias S. Keil (2), Shigang Yue (1), Jorge Cuadri-Carvajo (2), F. Claire Rind (1)1) School of Biology, Ridley Building, University of Newcastle upon Tyne, NE1 7RU, UK. 2) Instituto de Microelectronica de Sevilla (IMSE), Centro Nacional de Microelectronica (CNM),Avda. Reina Mercedes, 41012, Sevilla, Spain
Structure of the talk Introduction Improvements to the LGMD model
Light on and light off pathways Testing the LGMD model
Methods and Results Further improvements
Biologically inspired filtering of images by lateral inhibition
Analysing the filtered images using EMD like structures
Conclusions
Locusts as collision detectors The Lobula Giant Movement Detector
(LGMD) neuron responds most vigorously when objects of certain speeds and sizes approach, as if on a direct collision course
This has been linked to a predator avoidance, gliding behaviour in flying locusts
Predator avoidance caused by the LGMD
Angular subtense of object
LGMD Spikes
Inputs and structure of the LGMD
Why use the Locust LGMD to detect automotive collisions? Evolutionary honed collision avoidance
system Efficient circuit – based on insect
neurons Neural architecture well studied Responds optimally to imminent
collisions Simulated networks respond in a similar
manner to real locust
Limitations of existing model(e.g. Rind and Bramwell, 1996; Blanchard et al., 2000)
Simulations only tested in simple closed environment
Model needs to work in real automotive situations
Biology of the LGMD is not fully used – model only responds to change in light
Structure of the talk Introduction Improvements to the LGMD model
Light on and light off pathways Testing the LGMD model
Methods and Results Further improvements
Biologically inspired filtering of images by lateral inhibition
Analysing the filtered images using EMD like structures
Conclusions
Model Improvements – Light on and Light off Pathways
Small scale spatial antagonism between the pathways helps eliminate noise caused by vibration etc.
Larger scale antagonism can interfere with collision alerts
Model Improvements – Light on and Light off Pathways and Block Sum Cells
Input Image ‘S’ units – light on ~ light
off
Block Sum CellsAllow small scale antagonism ofpathways only
Location of BSC in model
Block sum cellsoccur here
Model Improvements - Block Sum Cells
Sum light on (-ve) and light off (+ve) excitationto obtain net excitation
Excitation (+ve only) is passed
to the LGMD
from theBSC
Block sum cells obtainexcitation from a 10x10section of the array of ‘S’ units
Light on andLight off excitationfrom ‘S’ units Block Sum Cells
LGMD
Structure of the talk Introduction Improvements to the LGMD model
Light on and light off pathways Testing the LGMD model
Methods and Results Further improvements
Biologically inspired filtering of images by lateral inhibition
Analysing the filtered images using EMD like structures
Conclusions
Testing the model in automotive situations
Input video sequences8 – 25 Hz
Input via frame-grabber of Playstation images8.3 Hz
Detecting collisions Membrane
potential of LGMD is obtained from sum of BSC
If a threshold is exceeded then the LGMD produces spikes
If > 2 spikes in 3 timesteps then collision detected
Results: LGMD model
Results show % of times collision was detected even if no collision occurred
Stationary car100 %
Moving Car90 %
Head on withmoving Car100 %
Entering Tunnel0 %
General Driving0 %
Driving in closeproximity0 %
Translating cars70 %
Why do translating cars proveproblematic?
• Excitation is much higher in the LGMD for translating objects• Locust LGMD ignores translating objects partially due to differences in mathematics of object approach
Structure of the talk Introduction Improvements to the LGMD model
Light on and light off pathways Testing the LGMD model
Methods and Results Further improvements
Biologically inspired filtering of images by lateral inhibition
Analysing the filtered images using EMD like structures
Conclusions
Image Filtering by LGMD network‘S’ units only excited by objects moving in close proximity to care.g.Colliding or translating objects
No threat Threat
Input Image
‘S’ units
Analysing the biologically filtered images Analysing patterns of excitation in ‘S’ or
‘BSC’ layers over time shows: No or little excitation – no threat. LGMD does not
reach threshold Excitation moving in one direction over time – no
threat, translating object. LGMD spikes can be suppressed
Excitation moving in all directions over time – collision threat, object on collision course is expanding in all directions. LGMD spikes and produces collision mitigation response
Structure of the talk Introduction Improvements to the LGMD model
Light on and light off pathways Testing the LGMD model
Methods and Results Further improvements
Biologically inspired filtering of images by lateral inhibition
Analysing the filtered images using EMD like structures
Conclusions
Incorporation of simple Elementary Movement Detector like units (EMDs) into the model
EMD like units take input from the Block Sum Cells – simplified visual environment
One detected ‘Right’ movement and one ‘Left’ movement
If membrane potential of ‘left’ EMDs was > 5 x potential of ‘right’ EMDs at time t or time t-1 then LGMD spikes were suppressed for time t, t+1 & t+2
Location of EMD like units
BSC
EMDsSuppressionof LGMD spikes
Results: LGMD incorporating EMDs
Results show % of times collision was detected even if no collision occurred
Stationary car85 %Was 100 %
Moving Car80 %Was 100 %
Head on withmoving Car50 %Was 100 %
Entering Tunnel0 %UnchangedGeneral Driving0 %Unchanged
Driving in closeproximity0 %UnchangedTranslating cars20 %Was 70 %
Results: LGMD and EMDs
Incorporation of EMDs reduce false collision alerts
Real collision detection was also reduced EMD model was very simple. Using a more
advanced (adaptive) model may improve the responses
Non bio-inspired image analysis could also be used on the biologically filtered ‘S’ units to improve model performance
Conclusions Locust based LGMD model can be used for
automotive collision detection In some situations modifications are
needed as the LGMD’s function in automotive situations is quite different to the evolved function in the locust
The biologically filtered image can be analysed to further assess the threat of collision
Acknowledgements Project funded by Future and
Emerging Technologies Grant from European Union (LOCUST – IST - 2002-38097)
We would like to thank Marrti Soininen of Volvo Car Corporation for supplying the video footage of car crashes
Other Improvements to the LGMD model On-Off cells look at absolute change in
image Lateral inhibition has a greater potential
spread to eliminate more non threatening situations
Spiking threshold of LGMD is self variable to allow a greater range of visually complex scenes to be investigated
Model parameters tuned using a Genetic Algorithm to automotive situations
Differences between automotive collisions and predator avoidance in locusts Locusts respond to
small, fast moving predators
Final excitation, just before predator strikes, is much higher
This can be used to distinguish between different object types
Small translating objects produce less excitation than larger objects
Recommended