Upload
bradley-walter
View
21
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Image Segmentation by Complex-Valued Units. Cornelius Weber Hybrid Intelligent Systems School of Computing and Technology University of Sunderland Presented at the Perceptual Dynamics Laboratory, RIKEN 8 th December 2005. Contents. • Attractor Network which Converges - PowerPoint PPT Presentation
Citation preview
Image Segmentation by Complex-Valued UnitsImage Segmentation by Complex-Valued Units
Cornelius WeberHybrid Intelligent Systems
School of Computing and TechnologyUniversity of Sunderland
Presented at the Perceptual Dynamics Laboratory, RIKEN 8th December 2005
ContentsContents
• Attractor Network which Converges
• Non-Convergence and Spike Synchrony
• Coupled Chaotic Oscillators for Spike Phases
• Outlook
ContentsContents
• Attractor Network which Converges
• Non-Convergence and Spike Synchrony
• Coupled Chaotic Oscillators for Spike Phases
• Outlook
Attractor Network:Attractor Network:Competition via RelaxationCompetition via Relaxation
Attractor Network:Attractor Network:Competition via RelaxationCompetition via Relaxation
weight profile rate profile
rate update
ri(t+1) = f (Σj wij rj(t))
winner
Response CharacteristicsResponse CharacteristicsResponse CharacteristicsResponse Characteristics
linear sparse competitive
Weber , C. Self-Organization of Orientation Maps, Lateral Connections, and Dynamic Receptive Fields in the Primary Visual Cortex. Proc. ICANN (2001)
Weber , C. Self-Organization of Orientation Maps, Lateral Connections, and Dynamic Receptive Fields in the Primary Visual Cortex. Proc. ICANN (2001)
Learning Object RecognitionLearning Object Recognition
attractor network Active units (features) not separated
Binding- and learning problem?green
redbackground
apple
Learning objects in cluttered background is difficult
Stringer, S.M. and Rolls, E.T. Position invariant recognition in the visual system with cluttered environments. Neural Networks 13, 305-15 (2000)
Stringer, S.M. and Rolls, E.T. Position invariant recognition in the visual system with cluttered environments. Neural Networks 13, 305-15 (2000)
ContentsContents
• Attractor Network which Converges
• Non-Convergence and Spike Synchrony
• Coupled Chaotic Oscillators for Spike Phases
• Outlook
Necker CubeNecker CubeNecker CubeNecker Cube
Attractor networks that minimize an energy function do not account for bi-stability
Neuronal Spike ChaosNeuronal Spike ChaosNeuronal Spike ChaosNeuronal Spike Chaos
A wide range of spiking neuron models displays three distinct categories of behavior:
- quiescence
- intense periodic seizure-like activity
- sustained chaos in normal operational conditions
Banerjee, A. On the Phase-Space Dynamics of Systems of Spiking Neurons. I: Model and Experiments. Neural Computation, 13(1), 161-93 (2001)
Banerjee, A. On the Phase-Space Dynamics of Systems of Spiking Neurons. I: Model and Experiments. Neural Computation, 13(1), 161-93 (2001)
Neuronal SynchronyNeuronal SynchronyNeuronal SynchronyNeuronal Synchrony
“cortical neurons often engage in oscillatory activity which is not stimulus locked but caused by internal interactions”
“activity synchronization was present in the expectation period before stimulus presentation and could not be induced de novo by the stimulus”
Singer, W. Synchronization, Bining and Expectancy. In: The Handbook of Brain Theory and Neural Networks, pp. 1136-43 (2003)
Singer, W. Synchronization, Bining and Expectancy. In: The Handbook of Brain Theory and Neural Networks, pp. 1136-43 (2003)
Cardoso de Oliviera, S., Thiele, A. and Hoffmann, K.P. Synchronization of neuronal activity during stimulus expectation in a direction discrimination task. J. Neurosci., 17, 9248-60 (1997)
Cardoso de Oliviera, S., Thiele, A. and Hoffmann, K.P. Synchronization of neuronal activity during stimulus expectation in a direction discrimination task. J. Neurosci., 17, 9248-60 (1997)
Neuronal Spike ChaosNeuronal Spike ChaosNeuronal Spike ChaosNeuronal Spike Chaos
We need a method to:
- create patterns of synchronization
- avoid long-term stabilization (bi-stability is welcome!)
van Leeuwen, C., Steyvers, M. and Nooter, M. Stability and Intermittency in Large-Scale Coupled Oscillator Models for Perceptual Segmentation. J. Mathematical Psychology, 41(4), 319-44 (1997)
van Leeuwen, C., Steyvers, M. and Nooter, M. Stability and Intermittency in Large-Scale Coupled Oscillator Models for Perceptual Segmentation. J. Mathematical Psychology, 41(4), 319-44 (1997)
ContentsContents
• Attractor Network which Converges
• Non-Convergence and Spike Synchrony
• Coupled Chaotic Oscillators for Spike Phases
• Outlook
Complex NumberComplex NumberComplex NumberComplex Number
φr r rate
φ phase
z = r eiφ
zi
1
= r cos φ + i r sin φ
Deterministic ChaosDeterministic ChaosDeterministic ChaosDeterministic Chaos
Logistic map:
Ф(t+1) = A Ф(t) (1- Ф(t))
Phase φ = 2π Ф
Coupling of the PhasesCoupling of the PhasesCoupling of the PhasesCoupling of the Phases
For phases: Σj wkj rj eiφ ≡ zkwf}
coupling strength for phasescomplex number
}
“Net input” to neuron k:
For rates: Σj wkj rj
j
weighted field
Relaxation of the PhasesRelaxation of the PhasesRelaxation of the PhasesRelaxation of the Phases
Compute “net input”: zkwf = Σj wkj rj eiφ
Compute new phase: Фk(t+1) = A Фkwf(t) (1- Фk
wf(t))
(remember: φ = 2π Ф)
From zwf, take phase φwf
j
Relaxation of Rates and PhasesRelaxation of Rates and PhasesRelaxation of Rates and PhasesRelaxation of Rates and Phases
Phase of any neuron behaves chaoticallyCoupled neurons have similar phases
Phase Separation HistogramPhase Separation HistogramPhase Separation HistogramPhase Separation Histogram
Large phase differences at boundary of activation hill
Toward Learning Object RecognitionToward Learning Object Recognition
attractor network
Toward Learning Object RecognitionToward Learning Object Recognition
attractor network
ContentsContents
• Attractor Network which Converges
• Non-Convergence and Spike Synchrony
• Coupled Chaotic Oscillators for Spike Phases
• Outlook
Plans and QuestionsPlans and QuestionsPlans and QuestionsPlans and Questions
- The higher hierarchical level shall benefit!
- Should the rates depend on the phases?
→ This would influence learning!
- Learning with Phase Timing Dependent Plasticity?