Upload
warren-bailey
View
218
Download
3
Tags:
Embed Size (px)
Citation preview
Maximum likelihood separation of spatially autocorrelated images
using a Markov model
Shahram Hosseini1, Rima Guidara1, Yannick Deville1
and Christian Jutten2
1. Laboratoire d’Astrophysique de Toulouse-Tarbes (LATT), Observatoire Midi-Pyrénées -Université Paul Sabatier Toulouse 3, 14 A. Edouard Belin, 31400
Toulouse, France. 2. Laboratoire des Images et des Signaux, UMR CNRS-INPG-UPS, 46 Avenue Félix
Viallet, 38031 Grenoble, France
MAXENT 2006, July 8-13, Paris-France 2
OUTLINE
Problem statement A maximum likelihood approach using Markov
model - Second-order Markov random field - Score function estimation - Gradient-based optimisation algorithm Experimental results Conlusion
MAXENT 2006, July 8-13, Paris-France 3
Problem statement
),( 21 nns ),(ˆ),( 2121 nnnn sy ),( 21 nnx 1ˆ ABA
Assumptions :
• Linear instantaneous mixture.
• K unknown independent source images, K observations, N=N1×N2 samples.
• Unknown mixing matrix A is invertible.
• Each source is spatially autocorrelated and can be modeled as a 2nd-order Markov random field.
Goal:
Compute B by a maximum likelihood (ML) approach.
Mixing matrix
Separating matrix
Problem statement A maximum likelihood approach using
Markov model Experimental results Conlusion
MAXENT 2006, July 8-13, Paris-France 4
Motivations Maximum likelihood approach: provides an asymptotically efficient
estimator (smallest error covariance matrix among unbiased estimators).
Modeling the source images by Markov random fields:
- Most of real images present a spatial autocorrelation within near pixels.
- Spatial autocorrelation can make the estimation of the model possible where the basic blind source separation methods cannot estimate it (if image sources are Gaussian but spatially autocorrelated).
- Markov random fields allow taking into account spatial autocorrelation without a priori assumption concerning the probability density of the sources.
MAXENT 2006, July 8-13, Paris-France 5
ML approach (1)
212111 1111 ,ΝΝ,...,x,ΝΝ,......,x,,...,x,xfC ΚΚx
K
iiisN NNssfC
i1
211
),(,...,1,1det
1
B
CMLB
B maxargˆ
Independence of sources
Problem statement A maximum likelihood approach using
Markov model Experimental results Conlusion
We denote the joint PDF of all the samples of all the components of the observation vector x.
Maximum likelihood estimate:
whereisf is the joint PDF of all the samples of
source si
MAXENT 2006, July 8-13, Paris-France 6
ML approach (2) Decomposition of the joint PDF of each source using Bayes rule Many possible sweeping trajectories that preserve continuity
and can exploit spatial autocorrelation within the image:
(1)
(2)
(3)
These different sweeping schemes being essentially equivalent, we chose the horizontal one.
MAXENT 2006, July 8-13, Paris-France 7
ML approach (3)
1,1,,1,,1,1,,,11,2
1,1,,1,1,11,1,2,13,11,12,11,1
,,...,1,1
21212
22
21
iiisiiis
iiisiiisiisis
iis
sNNsNNsfsNssf
sNsNsfsssfssfsf
NNssfF
ii
iiii
i
Bayes rule decomposition resulting from a horizontal sweeping:
To simplify F, sources are modeled by second-order Markov random fields Conditional PDF of a pixel given all remaining pixels of the image equals its conditional PDF given its 8 nearest neighbors.
),( 21 nns
MAXENT 2006, July 8-13, Paris-France 8
ML approach (4)
is the set of the predecessors of a pixel in the sense of the horizontal sweeping trajectory.
For a pixel not located on the boundary of the image, we obtain
211, ,\),(21
nlnknklksD inn Denote
21 ,nnD
1,1,,1,1,1,1,,)),(( 2121212121,21 21 nnsnnsnnsnnsnnsfDnnsf iiiiisnnis ii
If the image is quite large, pixels situated on the boundaries can be neglected. We can then write
1
1
2
22
1
22121212121 1,1,,1,1,1,1,,
N
n
N
niiiiis nnsnnsnnsnnsnnsfF
i
),( 21 nnsi
MAXENT 2006, July 8-13, Paris-France 9
ML approach (5) The initial joint PDF to be maximized:
1
1
2
22
1
2
.1
.N
n
N
nN NE
Taking the logarithm of C, dividing it by N and defining the spatial average operator
the log-likelihood can finally be written as
K
iiiiiisN nnsnnsnnsnnsnnsfEL
i1
21212121211 )1,1(),,1(),1,1(),1,(),((logdetlog B
212111 1111 ,ΝΝ,...,x,ΝΝ,......,x,,...,x,xfC ΚΚx
MAXENT 2006, July 8-13, Paris-France 10
ML approach (6)
),( 21 lnknsi
K
iiiiiisN
T nnsnnsnnsnnsnnsfEL
i1
21212121211 )1,1(),,1(),1,1(),1,(),(log
BB
B
Problem statement A maximum likelihood approach using
Markov model Experimental results Conlusion
Taking the derivative of L1with respect to the separating matrix B, we have
We define the conditional score function of the source si with respect to the pixel by:
),(
1,1,,1,1,1,1,,log,
21
212121212121
),(
lnkns
nnsnnsnnsnnsnnsfnnψ
i
iiiiislks
i
i
MAXENT 2006, July 8-13, Paris-France 11
Denoting:
- the column vector which has the conditional score fonctions of the K sources as components.
- the K-dimensional vector of observations.
we finally obtain
ML approach (7)
),( 21 nnx
)l,k(21
T21
)l,k(N
T1 ln,kn.n,nEL
xψBB s
),( 21),( nnlk
sΨ
)1,1(),0,1(),1,1(),1,0(),0,0( where
MAXENT 2006, July 8-13, Paris-France 12
Estimation of score functions
Conditional score fonctions must be estimated to solve our ML problem.
They may be estimated only via reconstructed sources yi(n1,n2). We used the method proposed in [D.-T.Pham, IEEE Trans. On Signal
Processing, Oct. 2004]
- A non-parametric kernel density estimator using third-order cardinal spline kernels.
- Estimation of joint entropies using a discrete Riemann sum.
Problem statement A maximum likelihood approach using
Markov model Experimental results Conlusion
No prior knowledge of the source distributions is needed.
Good estimation of the conditional score functions.
Very time consuming, especially for large-size images.
MAXENT 2006, July 8-13, Paris-France 13
An equivariant algorithm Initialize B=I.
Repeat until convergence : - Compute estimated sources y=Bx. Normalize to unit
power. - Estimate the conditional score functions - Compute the matrix
oldnew BGIB )(
),(2121
),( ,.,lk
TlkN lnknnnE yψG s
- Update B
Problem statement A maximum likelihood approach using
Markov model Experimental results Conlusion
MAXENT 2006, July 8-13, Paris-France 14
OUTLINE
Problem statement A maximum likelihood approach - Second-order Markov model - Score functions estimation - Gradient optimisation algorithm Experimental results Conclusion
MAXENT 2006, July 8-13, Paris-France 15
Experimental results Comparison with two classical methods: 1. SOBI algorithm - A second-order method - Joint diagonalisation of covariance matrices evaluated at
different lags. Exploits autocorrelation but ignores possible non-
Gaussianity2. Pham-Garat algorithm - A maximum likelihood approach - Sources are supposed i.i.d Exploits non-Gaussianity but ignores possible
autocorrelation.
Problem statement A maximum likelihood approach using
Markov model Experimental results Conlusion
MAXENT 2006, July 8-13, Paris-France 16
Artificial data (1) Generate two autocorrelated images :
1. Generate two independent white and uniformly distributed noise images
and .
2. Filter i.i.d noise images by 2 Infinite Impulse Response (IIR) filters
)1,1(3.0),1(5.0)1,1()1,(5.0),(),(
)1,1(3.0),1(5.0)1,1(4.0)1,(5.0),(),(
21221221222212212212
211211211211211211
nnsnnsnnsnnsnnenns
nnsnnsnnsnnsnnenns
ρ
)n,n(e 211 )n,n(e 212
Problem statement A maximum likelihood approach using
Markov model Experimental results Conlusion
In this case, generated images perfectly satisfy the working hypotheses : source images are stationary and second-order Markov random fields.
MAXENT 2006, July 8-13, Paris-France 17
Signal to Interference Ratio (SIR)
2
1i2
ii
2i
10 ])ys[(E
]s[Elog
2
1)dB(SIR
199.0
99.01A
Mixing matrix :
Problem statement A maximum likelihood approach using
Markov model Experimental results Conlusion
The mean of the SIR over 100 Monte Carlo simulations is computed and plotted as a function of the filter parameter ρ22 .
MAXENT 2006, July 8-13, Paris-France 18
Artificial data (2) Artificial data are generated by filtering i.i.d noise images by means of 2 Finite Impulse Response (FIR) Filters.
The source images are stationary but cannot be modeled by second-order Markov random fields.
The mean of the SIR over 100 Monte Carlo simulations is computed and plotted as a function of the selectivity of one of the filters.
MAXENT 2006, July 8-13, Paris-France 19
Astrophysical images (1)
13.0
3.011A
working hypotheses no longer true (non-stationary, non second-order Markov random field images)
199.0
99.012A
Weak mixture :
SIR=70 dB
Strong mixture:
Separation failed because of bad initial estimation of conditional score functions (estimated sources highly
different from actual sources).
Problem statement A maximum likelihood approach using
Markov model Experimental results Conlusion
MAXENT 2006, July 8-13, Paris-France 20
SIR Markov: 70 dB Pham-Garat: 13 dB SOBI: 36 dB
Solution : Initialize our method with a sub-optimal algorithm like SOBI to obtain a low mixture ratio.
MAXENT 2006, July 8-13, Paris-France 21
To conclude… A quasi-optimal maximum likelihood method taking into account
both non-Gaussianity and spatial autocorrelation is proposed.
Good performance on artificial and real data has been achieved.
Very time consuming : Solutions to reduce computational cost : - A parametric polynomial estimator of the conditional score
functions - A modified equivariant Newton optimization algorithm