Upload
federico-raue
View
341
Download
0
Tags:
Embed Size (px)
Citation preview
MotivationHMAX ModelImprovements
Summary
Biological Inspired Systemsapplied to Computer Vision
Federico Raue Rodriguez([email protected])
IUPR
July 2, 2012
This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License.
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
Contents
1 Motivation
2 HMAX Model
3 ImprovementsSparsityPooling MechanismInput
4 Summary
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
Contents
1 Motivation
2 HMAX Model
3 ImprovementsSparsityPooling MechanismInput
4 Summary
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
(slide from Fundamentals of AI – Prof. De Schreye (KULeuven))
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
Neuroscience may begin to provide new ideas and approachesto machine learning, AI and computer vision (Tomaso Poggio)
Interesting properties for visual recognition
a Invarianceb Specificity
Visual processing in cortex is classically modeled as a hierarchy
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
(slide from Learning in Hierarchical Architectures – Tomaso Poggio (McGovern
Institute))
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
(Perception Strategies in Hierarchical Vision Systems. (Wolf et al))
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
(slide from Learning in Hierarchical Architectures – Tomaso Poggio (McGovern
Institute))
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
Contents
1 Motivation
2 HMAX Model
3 ImprovementsSparsityPooling MechanismInput
4 Summary
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
Goal: Object categorization based on human visual system
Assumptions:
a Invariance to position and scaleb Feature specificity must be built up through separate
mechanismsc Extending the model of simple and complex cells of Hubel and
Wieseld Hierarchical feedforward architecturee Pooling mechanism
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
(slide from Learning in Hierarchical Architectures – Tomaso Poggio (McGovern
Institute))
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
(Hierarchical models of Object recognition in cortex (Riesenhuber et al.))
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
General Description of HMAX model
The standard model consists of four layers of computational unitswhere simple S units, which combine their inputs with Gaussian-liketuning to increase object selectivity, alternate with complex Cunits, which pool their inputs through maximum operation, therebyintroducing gradual invariance to scale and translation
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
(slide from Learning in Hierarchical Architectures – Tomaso Poggio (McGovern
Institute))
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
Simple Cells(S1) is a battery of Gabor filters
G (x , y) = exp
(−X 2 + γ2Y 2
2σ2
)× cos
(2π
λX
)Complex Cells(C1) show some tolerance to shift and size
a Larger receptive fieldsb Shape Invariance: respond to oriented bars or edges anywhere
within their receptive fieldc Scale Invariance: more broadly tuned to spatial frequency than
simple cells
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
Pooling operation from S1 to C1
S1 units: 16 scales arranged in 8 bands
For each orientation, it contains two S1 maps, two filter size
C1 responses: these maps are sub-sampled using a grid cell ofsize NΣ × NΣ (8x8)
From each grid cell we obtain one measurement by taking themaximum of all 64 elements
As a last stage we take a max over the two scales, byconsidering for each cell the maximum value from the twomaps
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
(Object Recognition with Features Inspired by Visual Cortex (Serre et al.))
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
(Object Recognition with Features Inspired by Visual Cortex (Serre et al.))
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
Learning Process
Large pool of K patches of various sizes at random positionsare extracted from a target set of images at the C1 level forall orientations
The patch size is n x n x 4 (The value 4 is due to 4orientations)
The training process ends by setting each of those patches asprototypes or centers of the S2 units, which behave as radialbasis function (RBF) units during recognition
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
Visual words in C2
When a new input is presented, each stored S2 unit isconvolved with the new (C1)Σ input image at all scales (thisleads to K x 8 (S2)Σ
i images), where the K factor correspondsto the K patches extracted during learning and the 8 factor,to the 8 scale bands
After taking a final max for each (S2)i map across all scalesand positions, we get the final set of K shift- andscale-invariant C2 units
The size of our final C2 feature vector thus depends only onthe number of patches extracted during learning and not nothe input image size
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
SparsityPooling MechanismInput
Contents
1 Motivation
2 HMAX Model
3 ImprovementsSparsityPooling MechanismInput
4 Summary
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
SparsityPooling MechanismInput
1 Extend the model using more biological information
Saliency ModelsNew Pooling mechanismRedefine the input image
2 Reduce the computational cost
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
SparsityPooling MechanismInput
Biological Motivation
Increase sparsity is to use a lateral inhibition model thateliminates weaker responses that disagree with the locallydominant ones
Our attention will be attracted to some locations mostlybecause their saliency, defined by contrasts in color, intensityor orientation
(Treisman) presented a theory about feature integration inhuman brain, which has two stages, the simple pre-attentionprocessing and complex attention processing. Some low levelfeatures will pop up automatically and generate the attentionarea in pre-attention processing
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
SparsityPooling MechanismInput
Computational Motivation
Simplifies structures and reduces computational costs
Feature or variable selection
Enhance the generalization ability of learning machines
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
SparsityPooling MechanismInput
(Multiclass Object Recognition with Sparse, Localized Features (Mutch and
Lowe)
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
SparsityPooling MechanismInput
|Fx(i)|+ |Fy(i)| ≥α
n
n∑k=1
(|Fx(k)|+ |Fy(k)|)
(Enhanced Biologically Inspired Model (Huang et al.))
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
SparsityPooling MechanismInput
(Enhanced Biologically Inspired Model (Huang et al.))
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
SparsityPooling MechanismInput
(Enhanced Biologically Inspired Model (Huang et al.))
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
SparsityPooling MechanismInput
1 Find the maximal response and its neighbors
2 Weak responses are removed due to inhibition effect3 New pooling Mechanism
a sum the energy of all responses remained by using differentweights for S1 units
C =1
NI0
∑xi ,yi∈I0
[wiS2(xi , yi )]
(Human age estimation using bio-inspired features (Guo et al.))b the STD operation is performed on the maximum map using a
cell grid of size Ns x Ns
std =
√√√√ 1
Ns × Ns
Ns×Ns∑i=1
(Fi − F̄
)2
(Enhanced Biologically Inspired Model (Huang et al.))
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
SparsityPooling MechanismInput
C =1
NI0
∑xi ,yi∈I0
[wiS2(xi , yi )]
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
SparsityPooling MechanismInput
(Enhanced Biologically Inspired Model (Huang et al.))
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
SparsityPooling MechanismInput
Relevant Component Analysis (RCA): finds a linear embeddingtransformation that minimizes the distances between points
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
SparsityPooling MechanismInput
Complement using dorsal stream (where)
Cells respond to colored stimuli more strongly than colorlessone in the Inferior Temporal (IT) and the visual areas V4 ofthe visual cortex
Analogous to the ’center-on surround-off’ center surroundprocessing that occurs in the retina and in the lateralgeniculate nucleus (LGN)
Some region (in the brain) are more active for face imageswhen compared to images of other objects
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
SparsityPooling MechanismInput
Hierarchical: gradually increase both the selectivity of neuronsalong with their invariance to 2D transformationsHypothesis: neurons in intermediate visual areas of the dorsalstream such as MT, MST and higher polysensory areas aretuned to spatio-temporal features of intermediate complexity,which pool over afferant input units tuned to differentdirections of motion
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
SparsityPooling MechanismInput
S1 Units
Gray-value video sequence at all positionThree different types of S1
a Space-time gradient-based: Space and time gradients
| ItIx + 1
| | ItIy + 1
|
b Optical flow based S1 units: Optical flow of the input usingLucas & Kanade’s alg.
b(θ, θp) = {1
2[1 + cos(θ − θp)]}q × exp(−|v − vp|)
4 directions and two speedsc Space-time oriented S1 units:
Add a temporal dimension to their receptive fields3rd derivatives fo Gaussians8 space-time filters tuned to 4 directions and 2 speedsSize of receptive fields was 9(pixels)x9(pixels)x9(frames)
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
SparsityPooling MechanismInput
Proposed by Plebe et al
Extract visual attribute for object recognition based oninfant’s brain
Children between 8 and 10 months old, their objectcategorization model is stable and flexible
10-month-old infants ’are sensitive to social cues but cannotrecruit them for word learning’
Early vocabulary is made up of the objects infants mostfrequency see
Connectionist model with backpropagation developed ageneral model based on similarities without taking intoaccount physiological and cognitive constraints
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
SparsityPooling MechanismInput
Implementation based on Laterally InterconnectedSynergetically Self Organizing Map architecture (LISSOM)
Hebbian Law: explains the adaptation of neurons in the brainduring the learning process“. . . , that any two cells or systems of cells that arerepeatedly active at the same time will tend to becomeassociated, so that activity in one facilitates activity in theother.”
Two paths: one for visual and the other for auditory channel
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
SparsityPooling MechanismInput
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
SparsityPooling MechanismInput
Exposure to stimuli
Visual path in the model develops in two stages.1 Random blobs, simulating pre-natal waves of spontaneous
activity, known to be essential in the early development of thevisual system
2 Natural images are used (After eye opening)
Auditory path there are different stages1 Random patches in frequency-time domain, with shorter
duration for HPC and longer for LPC2 7200 most common English words (lengths between 3 and 10
characters)
Last stage: an object is viewed and a word corresponding toits basic category is heard simultaneouly
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
SparsityPooling MechanismInput
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
SparsityPooling MechanismInput
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
Contents
1 Motivation
2 HMAX Model
3 ImprovementsSparsityPooling MechanismInput
4 Summary
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision
MotivationHMAX ModelImprovements
Summary
Biological models are suitable features for visual recognition
RobustInvariance
Two Pathways1 Ventral stream (What?)2 Dorsal stream (Where?)
Depending on the task HMAX model changes
Parameters (Aging Detection)Pooling function (Energy model, Standard Deviation)
Federico Raue Rodriguez ([email protected]) Biological Inspired Systems applied to Computer Vision