View
0
Download
0
Category
Preview:
Citation preview
物理フラクチュオマティクス論Physical Fluctuomatics
応用確率過程論A li d St h ti PApplied Stochastic Process
第5回グラフィカルモデルによる確率的情報処理5th Probabilistic information processing by means of5th Probabilistic information processing by means of
graphical model
東北大学 大学院情報科学研究科 応用情報科学専攻田中 和之(Kazuyuki Tanaka)k @ i i t h k jkazu@smapip.is.tohoku.ac.jp
http://www.smapip.is.tohoku.ac.jp/~kazu/
1物理フラクチュオマティクス論(東北大)
今回の講義の講義ノート
田中和之著:確率モデルによる画像処理技術入門,
森北出版,2006.
2物理フラクチュオマティクス論(東北大)
ContentsContents
11 IntroductionIntroduction1.1. IntroductionIntroduction2.2. Probabilistic Image ProcessingProbabilistic Image Processing3.3. Gaussian Graphical ModelGaussian Graphical Model44 Statistical Performance AnalysisStatistical Performance Analysis4.4. Statistical Performance AnalysisStatistical Performance Analysis5.5. Concluding RemarksConcluding Remarks
物理フラクチュオマティクス論物理フラクチュオマティクス論((東北大東北大)) 33
ContentsContents
11 IntroductionIntroduction1.1. IntroductionIntroduction2.2. Probabilistic Image ProcessingProbabilistic Image Processing3.3. Gaussian Graphical ModelGaussian Graphical Model44 Statistical Performance AnalysisStatistical Performance Analysis4.4. Statistical Performance AnalysisStatistical Performance Analysis5.5. Concluding RemarksConcluding Remarks
物理フラクチュオマティクス論物理フラクチュオマティクス論((東北大東北大)) 44
Markov Random Fields for Image Processing
Markov Random Fields
S G d D G (1986) IEEE T ti PAMIS G d D G (1986) IEEE T ti PAMI
Markov Random Fields are One of Probabilistic Methods for Image processing.
S. Geman and D. Geman (1986): IEEE Transactions on PAMIS. Geman and D. Geman (1986): IEEE Transactions on PAMIImage Processing for Image Processing for Markov Random Fields (MRF)Markov Random Fields (MRF)(Simulated Annealing Line Fields)(Simulated Annealing Line Fields)(Simulated Annealing, Line Fields)(Simulated Annealing, Line Fields)
J Zhang (1992): IEEE Transactions on Signal ProcessingJ Zhang (1992): IEEE Transactions on Signal ProcessingJ. Zhang (1992): IEEE Transactions on Signal ProcessingJ. Zhang (1992): IEEE Transactions on Signal ProcessingImage Processing in EM algorithm for Image Processing in EM algorithm for Markov Markov Random Fields (MRF)Random Fields (MRF) (Mean Field Methods)(Mean Field Methods)Random Fields (MRF)Random Fields (MRF) (Mean Field Methods)(Mean Field Methods)
物理フラクチュオマティクス論(東北大) 5
Markov Random Fields for Image ProcessingIn Markov Random Fields, we have to consider not only the states with high probabilities but also ones with low probabilities.I M k R d Fi ld
Hyperparameter EstimationIn Markov Random Fields, we
have to estimate not only the image but also hyperparameters
Estimation
image but also hyperparameters in the probabilistic model.We have to perform the
Statistical Quantitiesp
calculations of statistical quantities repeatedly.
Estimation of Image
We can calculate statistical quantities by adopting the Gaussian graphical model as a prior probabilistic model
d b i G i i t l f l物理フラクチュオマティクス論(東北大) 6
and by using Gaussian integral formulas.
Purpose of My TalkR i f f l ti f b bili tiReview of formulation of probabilistic model for image processing by means of conventional statistical schemes.Review of probabilistic image processing by using Gaussian graphical model (Gaussian Markov Random Fields) as the(Gaussian Markov Random Fields) as the most basic example.
K Tanaka: StatisticalK Tanaka: Statistical Mechanical ApproachMechanical ApproachK. Tanaka: StatisticalK. Tanaka: Statistical--Mechanical Approach Mechanical Approach to Image Processing (Topical Review), J. Phys. to Image Processing (Topical Review), J. Phys. A: Math. Gen., vol.35, pp.R81A: Math. Gen., vol.35, pp.R81--R150, 2002.R150, 2002.A: Math. Gen., vol.35, pp.R81A: Math. Gen., vol.35, pp.R81 R150, 2002.R150, 2002.
Section 2 and Section 4 are summarized in the present talk.
物理フラクチュオマティクス論(東北大) 7
ContentsContents
11 IntroductionIntroduction1.1. IntroductionIntroduction2.2. Probabilistic Image ProcessingProbabilistic Image Processing3.3. Gaussian Graphical ModelGaussian Graphical Model44 Statistical Performance AnalysisStatistical Performance Analysis4.4. Statistical Performance AnalysisStatistical Performance Analysis5.5. Concluding RemarksConcluding Remarks
物理フラクチュオマティクス論物理フラクチュオマティクス論((東北大東北大)) 88
Image Representation in Computer Vision
Digital image is defined on the set of points arranged on a square lattice.qThe elements of such a digital array are called pixels.We have to treat more than 100,000 pixels even in the , pdigital cameras and the mobile phones.
xx )1,1( )1,2( )1,3(
)2,1( )2,2( )2,3(
y y )3,1( )3,2( )3,3(Pixels200307480640
物理フラクチュオマティクス論(東北大) 9
y y),( yx
Pixels200,307480640
Image Representation in Computer Vision
x yxfyx ,),(
0, yxf 255yxf
Pixels 65536256256 yAt each point the intensity of light is represented as an
0, yxf 255, yxf
At each point, the intensity of light is represented as an integer number or a real number in the digital image datadata.A monochrome digital image is then expressed as a two-dimensional light intensity function and the value isdimensional light intensity function and the value is proportional to the brightness of the image at the pixel.
物理フラクチュオマティクス論(東北大) 10
Noise Reduction by Conventional FiltersNoise Reduction by Conventional FiltersConventional FiltersConventional Filters
The function of a linear filter is to take the sumfilter is to take the sum of the product of the mask coefficients and the
Smoothing Filters
mask coefficients and the intensities of the pixels.
173120219202190202192
A
192 202 190
202 219 120
192 202 190
202 173 120
Smoothing Filters
173110218100120219202Average
100 218 110 100 218 110
It is expected that probabilistic algorithms for p p gimage processing can be constructed from such aspects in the conventional signal processing.
物理フラクチュオマティクス論(東北大) 11
Markov Random Fields Probabilistic Image ProcessingAlgorithm
Bayes Formula and Bayesian Network
}Pr{}|Pr{}|P { AABBAPrior
}Pr{}Pr{}|Pr{}|Pr{
BAABBA Probability
Posterior Probability
}{
Bayes Rule AData-Generating Process
Posterior Probability Bayes RuleEvent A is given as the observed data.Event B corresponds to the original
A
BEvent B corresponds to the original information to estimate. Thus the Bayes formula can be applied to the
BBayesianThus the Bayes formula can be applied to the
estimation of the original information from the given data.
Bayesian Network
物理フラクチュオマティクス論(東北大) 12
Image Restoration by Probabilistic Model
NoiseAssumption 1: The degraded image is randomly generated
Transmission
Noise g y gfrom the original image by according to the degradation
Original Image
Degraded Image
Transmission process. Assumption 2: The original image is randomly generatedImage Image
Posterior
image is randomly generated by according to the prior probability.
PriorLikelihood
}Image Degraded|Image OriginalPr{
}ageDegradedImPr{}Image OriginalPr{}Image Original|Image DegradedPr{
物理フラクチュオマティクス論(東北大) 13
Likelihood Marginal Bayes Formula
Image Restoration by Probabilistic ModelThe original images and degraded images areThe original images and degraded images are represented by f = {fi} and g = {gi}, respectively.
DegradedOriginal
ImageImage
),( iii yxr
Position Vectori
Position Vector of Pixel i i
fi: Light Intensity of Pixel iin Original Image
gi: Light Intensity of Pixel iin Degraded Image
物理フラクチュオマティクス論(東北大) 14
in Original Image in Degraded Image
Probabilistic Modeling of Image Restoration
Posterior
PriorLikelihood
}Image Degraded|Image OriginalPr{
}ImageOriginalPr{}ImageOriginal|ImageDegradedPr{
Assumption 1: A given degraded image is obtained from the original image by changing the state of each pixel to another state by the same probability, independently of the other pixels.
}ImageOriginal|ImageDegradedPr{ fggi gi
Vi
ii fg ),(}|Pr{ }gg|gg{
fFgGfg
Random Fieldsfi fi
or
物理フラクチュオマティクス論(東北大) 15
Random Fields
Probabilistic Modeling of Image RestorationImage Restoration
Posterior
}ImageDegraded|ImageOriginalPr{
PriorLikelihood
}ImageOriginalPr{}ImageOriginal|ImageDegradedPr{
}ImageDegraded|ImageOriginalPr{
}ImageOriginalPr{}ImageOriginal|ImageDegradedPr{
Assumption 2: The original image is generated according to a prior probability Prior Probability consists of a product of
}IO i i lP { f
prior probability. Prior Probability consists of a product of functions defined on the neighbouring pixels.
Eij
ji ff ),(}Pr{ }ImageOriginalPr{
fFf
i jEij
Random FieldsProduct over All the Nearest Neighbour Pairs of Pixels
物理フラクチュオマティクス論(東北大) 16
Product over All the Nearest Neighbour Pairs of Pixels
Prior Probability for Binary ImageIt i i t t h h ld th f tiIt is important how we should assume the function (fi,fj) in the prior probability. 1,0if
Eijji ff ),(}Pr{ fF
i jj
We assume that every nearest-neighbour pair of pixels take the same state of each other in the prior probability.
)0,1()1,0()0,0()1,1(
== >p p p
21 p
21i j Probability of
Neigbouring Pi l
物理フラクチュオマティクス論(東北大) 17
> Pixel
Prior Probability for Binary Image
== >p p p
21 p
21i j Probability of
Nearest Neigbour Pair of Pixels
Which state should the center ?
Pair of Pixels
pixel be taken when the states of neighbouring pixels are fixed g g pto the white states?
>
Prior probability prefers to the configuration with the least number of red lines
物理フラクチュオマティクス論(東北大) 18
with the least number of red lines.
Prior Probability for Binary ImagePrior Probability for Binary Imagey y gy y g
== >p p
Which state should the center
?-?== >
Which state should the center pixel be taken when the states of neighbouring pixels are fixed as g g pthis figure?
> >=
Prior probability prefers to the configuration with the least number of red lines
物理フラクチュオマティクス論(東北大) 19
with the least number of red lines.
What happens for the case of large umber of pixels?large umber of pixels?
1.0
Co ariance Patterns with
0 4
0.6
0.8Covariance between the nearest neghbour
Patterns with both ordered statesand disordered states
p 0.0
0.2
0.4
0 0 0 2 0 4 0 6 0 8 1 0
nearest neghbour pairs of pixels are often generated
near the critical point. 0.0 0.2 0.4 0.6 0.8 1.0lnpsmall p large p
Sampling by Marko chain
Disordered State Critical Point(Large fluctuation)
Marko chain Monte Carlo
Ordered State
物理フラクチュオマティクス論(東北大) 20
(Large fluctuation)
Pattern near Critical Point f P i P b biliof Prior Probability
1.0Covariance We regard that patterns
0 4
0.6
0.8Covariance between the nearest neghbour
g pgenerated near the critical point are similar to the
0.0
0.2
0.4
0 0 0 2 0 4 0 6 0 8 1 0l
pairs of pixels local patterns in real world images.
0.0 0.2 0.4 0.6 0.8 1.0ln psmall p large p
similar
物理フラクチュオマティクス論(東北大) 21
similar
ContentsContents
11 IntroductionIntroduction1.1. IntroductionIntroduction2.2. Probabilistic Image ProcessingProbabilistic Image Processing3.3. Gaussian Graphical ModelGaussian Graphical Model44 Statistical Performance AnalysisStatistical Performance Analysis4.4. Statistical Performance AnalysisStatistical Performance Analysis5.5. Concluding RemarksConcluding Remarks
物理フラクチュオマティクス論物理フラクチュオマティクス論((東北大東北大)) 2222
Bayesian Image Analysis by Gaussian Graphical ModelGaussian Graphical Model
11
,if
Ejiji ff
Z },{
2
PR 21exp
)(1|Pr
fFPriorProbability Probability
V:Set of all the pixels
||21},{
2PR 2
1exp)( VEji
ji dzdzdzzzZ
0005.0 0030.00001.0
Patterns are generated by MCMC.the pixels
Markov Chain Monte Carlo Method
E:Set of all the nearest-neighbour pairs of pixels
物理フラクチュオマティクス論(東北大) 23
Markov Chain Monte Carlo Methodp p
Bayesian Image Analysis by Gaussian Graphical ModelGaussian Graphical Model
,, ii gfDegradation Process is assumed to be the additive white Gaussian noise.
Viii gf 2
22 21exp
2
1,Pr
fFgGVi 2
V: Set of all the pixels
Original Image f Gaussian Noise n Degraded Image g
2,0~ Nfg ii Degraded image is obtained by adding a ,Nfg ii
Hi f G i R d N b
y gwhite Gaussian noise to the original image.
物理フラクチュオマティクス論(東北大) 24
Histogram of Gaussian Random Numbers
Bayesian Image Analysis
f fF Pr fFgG Pr gfg
fF Pr fFgG Pr gOriginalImage
Degraded Image
Prior Probability Degradation Process
PrPrPr
fFfFgGgGfF
Image g
Posterior Probability
PrPr
gGgGfF
Ejiji
Viii ffgf
},{
),(),(
E:Set of all
V:Set of All the pixels
Image processing is reduced to calculations of averages, variances and co-variances in the posterior probability
E:Set of all the nearest neighbour
i f i l
物理フラクチュオマティクス論(東北大) 25
posterior probability.pairs of pixels
Estimation of Original Image
We have some choices to estimate the restored image from posterior probability. p p y
In each choice, the computational time is generally exponential order of the number of pixels.
}|Pr{maxargˆ gGzFfz
Maximum A Posteriori (MAP) estimation
(1)
}|Pr{maxargˆ gG iiz
i zFfi
Maximum posterior marginal (MPM) estimation
(2)
if
ii fF\
}|Pr{}|Pr{f
gGfFgG
2}|Pr{minargˆ zgGzF dzf iiii
Thresholded Posterior Mean (TPM) estimation
(3)
物理フラクチュオマティクス論(東北大) 26
i Mean (TPM) estimation
Statistical Estimation of HyperparametersHyperparameters are determined so as to maximize the marginal likelihood Pr{G=g|,} with respect to
},|Pr{max arg)ˆ,ˆ( gG
respect to ,
zzFzFgGgG d}|Pr{},|Pr{},|Pr{
,
gg }|{},|{},|{
Original Image Degraded ImageVy
f g
Marginalized with respect to F
}|Pr{ fF },|Pr{ fFgG gy
x Marginalized with respect to F},|Pr{ gG
物理フラクチュオマティクス論(東北大) 27
Marginal Likelihood
Bayesian Image Analysis ,, ji gf
A Posteriori Probability
},|Pr{
}|Pr{},|Pr{},,|Pr{gG
fFfFgGgGfF
}{
222
POS 21
21exp
,,1g
ffgfZ Eji
jiVi
ii
TT2
},{POS
21))((
21exp1
,,
fCfgfgfg
g
Z
EjiVi
POS 22,,g Z
||21
TT2POS
21))((
21exp,, VdzdzdzZ zCzgzgzg
Gaussian Graphical Model
22
物理フラクチュオマティクス論(東北大) 28
Average of Posterior Probability
gGzFzF ||21},,|Pr{,
Vdzdzdz
CI
IzCICI
Izz ||21
T
22
22 2
1exp
Vdzdzdzgg
I
CIIzCI
CIIz ||21
T
22
22 2
1exp
Vdzdzdzgg
gCI
I2
)0,00())()((21exp)(
||
||21T2
V
V ,,dzdzdz
mzCI mz mz
Gaussian Integral formula
物理フラクチュオマティクス論(東北大) 29
Bayesian Image Analysis by Gaussian Graphical Model by Gauss a G ap ca ode ,, ii gf
fff 22 11}|{ Gf
Posterior Probability
Ejiji
Viii ffgf
},{
222 2
12
1exp},,|Pr{
gGfF
V S t f llAverage of the posterior probability can be calculated by using the multi- E:Set of all the
V:Set of all the pixels
zgGzFzf },,|Pr{ˆ d
dimensional Gauss integral Formula
|V|x|V| matrix
nearest-neghbour pairs of pixels
gCI
I
gf
2
},,|{
},{1
4Eji
jiji C
|V|x|V| matrix
CI otherwise0
Multi Dimensional Gaussian Integral Formula
物理フラクチュオマティクス論(東北大) 30
Multi-Dimensional Gaussian Integral Formula
Statistical Estimation of Hyperparameters
)()2(
),,(}|Pr{},|Pr{},|Pr{ 2/||2
POS
Z
Zd V
gzzFzFgGgG )()2( PR
2/||2 ZV
POS ,,Z g
||21
TT2
21))((
21exp Vdzdzdz zCzgzgz
O i i l I D d d I
||21
TPR
21exp VdzdzdzZ zCz
f g}|Pr{ fF },|Pr{ fFgG gOriginal Image Degraded ImageVy
Marginalized with respect to Fx
},|Pr{ gG
物理フラクチュオマティクス論(東北大) 31
Marginal Likelihood},|Pr{ gG
C l l ti f P titi F tiCalculations of Partition Function
Czz )2(1exp)(||
||21V
VPR dzdzdzZ
T
POS ),,(g Z
CCzz
det2exp)( ||||21 VVPR dzdzdzZ
T
||21T
2
T
22
22
1exp
21
21exp
gCg
gCI
CggCI
IzCIgCI
Iz
Vdzdzdz
||21
T
22
22
2
2
1exp
2exp
gCI
IzCIgCI
Iz
gCI
g
Vdzdzdz
T22
||2
21exp
)det()2(
2
gCI
CgCI
CICI
V
AmzA mz
det)2()( )(
21exp
||||21
TV
Vdzdzdz
物理フラクチュオマティクス論(東北大) 32
(A is a real symmetric and positive definite matrix.)Gaussian Integral formula
Exact expression of Marginal Likelihood in G i G iGaussian Graphical Model
)()2(
),,(}|Pr{},|Pr{},|Pr{
PR2/||2
POS
Z
Zd V
gzzFzFgGgG
Multi-dimensional Gauss integral formula
T
22 21exp
det2
det},|Pr{ gCI
CgCI
CgG
V det2 CICI
y }|P {ˆˆ GWe can construct an exact EM algorithm.
If̂
Vy
},|Pr{max arg,
,
gG
物理フラクチュオマティクス論(東北大) 33
gCI
f 2 x
Bayesian Image Analysis by Gaussian Graphical Modelby Gaussian Graphical Model
1
Iteration Procedure in Gaussian Graphical Modelg
1T
22
2
11||1
111Tr
||1
g
CICg
CIC
ttVttt
Vt
2422
g
T22
242
2
2
11
11||
111
1Tr||
1 gCI
CgCI
I
tt
ttVtt
tV
t
I 2ˆ f̂gCI
Im 2)()()(
ttt
2)(minarg)(ˆ tmztf ii
zi
i f
0.0006
0.0008
0.001 t
0
0.0002
0.0004
tg ˆ物理フラクチュオマティクス論(東北大) 34
0 20 40 60 80 100 tg f
Image Restoration by Markov Random Field Model and Conventional FiltersField Model and Conventional Filters
MSEStatistical Method 315
Lowpass Filter
(3x3) 388(5x5) 413
Original ImageOriginal Image Degraded ImageDegraded Image
Filter (5x5) 413
Median Filter
(3x3) 486
(5x5) 445
2ˆ||
1MSE
Vi
ii ffV
( )
RestoredRestoredImageImage
V:Set of all the pixels
(3x3) Lowpass(3x3) Lowpass (5x5) Median(5x5) MedianMRFMRF
物理フラクチュオマティクス論(東北大) 35
(3x3) Lowpass(3x3) Lowpass (5x5) Median(5x5) MedianMRFMRF
ContentsContents
11 IntroductionIntroduction1.1. IntroductionIntroduction2.2. Probabilistic Image ProcessingProbabilistic Image Processing3.3. Gaussian Graphical ModelGaussian Graphical Model44 Statistical Performance AnalysisStatistical Performance Analysis4.4. Statistical Performance AnalysisStatistical Performance Analysis5.5. Concluding RemarksConcluding Remarks
物理フラクチュオマティクス論物理フラクチュオマティクス論((東北大東北大)) 3636
Performance Analysis
Sample Average of Mean Square Error
1y
1x̂
lity
e e
2y
2x̂
ˆroba
bil
e W
hit
an N
ois
x 3y
4y3x
4x̂erio
r Pr
Add
itiv
Gau
ssia
4
5y
4
5x̂Post
e
SignalA G
Estimated ResultsObserved
Data 5
1
2||ˆ||51][MSE nxx
物理フラクチュオマティクス論(東北大) 37
15 n
Statistical Performance AnalysisStatistical Performance Analysis
ydxXyYxyhV
x },|Pr{),,(1),|(MSE2
V
Original Image Degraded Image
gy
Additive White Gaussian Noise
},|Pr{ xXyY x
),,( yh },,|Pr{ yYxX
Restored Image
},|Pr{ xXyY
g
Posterior ProbabilityAdditive White Gaussian Noise
物理フラクチュオマティクス論(東北大) 38
Statistical Performance Analysis
ydxXyYxyhV
x Pr,,1),|MSE(2
ydxXyYxyV
V
Pr1 212 CI
V
otherwise,0
},{,1,4
EjiVji
ji C
xdyxPxyh
12
)|(,,
CI
1exp,Pr 2yxxXyY ii
,
y2 CI
)2
1exp(
2exp,Pr
22
2
yx
yxxXyYVi
ii
物理フラクチュオマティクス論(東北大) 39
2
Statistical Performance Estimation for Gaussian Markov Random Fields
21
22
||2
2
2
21exp
211
},|Pr{),,(1),|(MSE
ydyxxyV
ydxXyYxyhV
x
V
CII
otherwise,0
},{,1,4
EjiVji
ji C
2||22
22
||2
22
111
21exp
21)(1
22
ydxyxxxyV
V
V
V
CI
CII
CII
CI
22
||
2
2
2
T
2
2
2
22
||
2
2
2
21exp
21)()(1
21exp
21)(1
ydxyxxyxxyV
ydxyxxyV
V
CIC
CII
CIC
CII
CIC
CII
||2
22
||
22T
111
21exp
21)(
)()(1
22
ydxyxyxyV
V
V
V
C
CII
CICICICI
22
||
22
2T
22
||
22
2T
21exp
21)(
)(1
21exp
21
)()(1
ydxyxyxV
ydxyxxyV
V
CIC
CIC
= 0
2422
22
||
22
242T
11
21exp
21
)(1
22)(
ydxyxxV
VV
CI
CIC
CI
物理フラクチュオマティクス論(東北大) 40
T2222 )(||
1)(
Tr1 xI
xVV
CC
CII
Statistical Performance Estimation for Gaussian Markov Random Fields
ydxXyYxyhV
x 2 },|Pr{),,(1),|(MSE
ydyxxyV
VV
22
||
2
2
2 21exp
211
CII
xI
xVV
V
22
242T
22
2
)(||1
)(Tr1
22
CC
CII
CI
},{,1,4
EjiVji
ji C otherwise,0
600=40
600=40
),|(MSE x ),|(MSE x
200
400=40
200
400=40
0
200
0 0 001 0 002 0 0030
200
0 0 001 0 002 0 003
物理フラクチュオマティクス論(東北大) 41
0 0.001 0.002 0.0030 0.001 0.002 0.003
ContentsContents
11 IntroductionIntroduction1.1. IntroductionIntroduction2.2. Probabilistic Image ProcessingProbabilistic Image Processing3.3. Gaussian Graphical ModelGaussian Graphical Model44 Statistical Performance AnalysisStatistical Performance Analysis4.4. Statistical Performance AnalysisStatistical Performance Analysis5.5. Concluding RemarksConcluding Remarks
物理フラクチュオマティクス論物理フラクチュオマティクス論((東北大東北大)) 4242
Summary
Formulation of probabilistic model for image processing by means ofimage processing by means of conventional statistical schemes has been summarized.Probabilistic image processing by usingProbabilistic image processing by using Gaussian graphical model has been shown as the most basic exampleas the most basic example.
物理フラクチュオマティクス論(東北大) 43
ReferencesReferences
K. Tanaka: Introduction of Image Processing by P b bili ti M d l M ikit P bli hi CProbabilistic Models, Morikita Publishing Co., Ltd., 2006 (in Japanese) .
K T k St ti ti l M h i l A h tK. Tanaka: Statistical-Mechanical Approach to Image Processing (Topical Review), J. Phys. A, 35 (2002)35 (2002).
A. S. Willsky: Multiresolution Markov Models for Signal and Image Processing Proceedings ofSignal and Image Processing, Proceedings of IEEE, 90 (2002).
物理フラクチュオマティクス論(東北大) 44
Problem 5-1: Derive the expression of the posterior probability Pr{F=f|G=g,,} by using p p y { f| g } y gBayes formulas Pr{F=f|G=g,,}=Pr{G=g|F=f,}Pr{F=f,}/Pr{G=g|,}. Here P {G |F f } d P {F f } d t bPr{G=g|F=f,} and Pr{F=f,} are assumed to be as follows:
gf 21exp1Pr fFgG
Vi
ii gf22 2exp
2,Pr
fFgG
ji ff
Z2
21exp
)(1|Pr
fF
Eji
jZ },{PR 2)(
||212
PR 21exp)( Vji dzdzdzzzZ
},{2 Eji
[Answer]
Ejiji
Viii ffgf
Z },{
222
POS 21
21exp
,,1},,|Pr{
ggGfF
22 11 ddd
物理フラクチュオマティクス論(東北大) 45
||21
},{
222PR 2
12
1exp VEji
jiVi
ii dzdzdzzzgzZ
Problem 5-2: Show the following equality.
},{
222 2
12
1
Eji
jiVi
ii ffgf
TT2
},{
21))((
21 fCfgfgf
j
22
4 ji
otherwise0},{1 Eji
jji C
物理フラクチュオマティクス論(東北大) 46
otherwise0
Problem 5-3: Show the following equality.
TT2
21))((
21 zCzgzgz
T
22
22 1 gIzCIgIz
T
222
12
gCg
CICI
22g
CIg
物理フラクチュオマティクス論(東北大) 47
Problem 5-4: Show the following equalities by using theProblem 5 4: Show the following equalities by using the multi-dimensional Gaussian integral formulas.
||2
||21},{
222POS
1)2(
21
21exp,,
C
g
V
VEji
jiVi
ii dzdzdzffgfZ
T
22
||2
21exp
)det()2( g
CICg
CI
V
Cdet
)2(21exp)( ||
||
||21},{
2V
V
VEji
jiPR dzdzdzffZ
物理フラクチュオマティクス論(東北大) 48
P bl 5 5 D i h di i f hProblem 5-5: Derive the extremum conditions for the following marginal likelihood Pr{G=g} with respect to the hyperparameters and the hyperparameters and .
T
221expdet},|Pr{ gCgCgG
V
22 2
pdet2
},|{ gCI
gCI
g V
T2 111 CC
[Answer]
T
22 ||1Tr
||11 g
CICg
CIC
VV
2422 11 CI
T22
242
2
22
||1Tr
||1 g
CI
CgCI
I
VV
物理フラクチュオマティクス論(東北大) 49
P bl 5 6 D i h di i f hProblem 5-6: Derive the extremum conditions for the following marginal likelihood Pr{G=g} with respect to the hyperparameters and the hyperparameters and .
T
221expdet},|Pr{ gCgCgG
V
22 2
pdet2
},|{ gCI
gCI
g V
T2 111 CC
[Answer]
T
22 ||1Tr
||11 g
CICg
CIC
VV
2422 11 CI
T22
242
2
22
||1Tr
||1 g
CI
CgCI
I
VV
物理フラクチュオマティクス論(東北大) 50
Problem 5-7: Make a program that generate a degraded image by the additive white Gaussian noise. Generate some degraded images from a given standard images by settingdegraded images from a given standard images by setting =10,20,30,40 numerically. Calculate the mean square error (MSE) between the original image and the degraded image.( ) g g g g
21MSE gf
Original Image Gaussian Noise Degraded Image
||
MSE
Vi
ii gfV
Original Image Degraded Image
K.Tanaka: Introduction of Image Processing by Probabilistic Models
Hi t f G i R dSample Program:
Processing by Probabilistic Models, Morikita Publishing Co., Ltd., 2006 .
物理フラクチュオマティクス論(東北大) 51
Histogram of Gaussian Random Numbers Fi Gi~N(0,402)
http://www.morikita.co.jp/soft/84661/
Problem 5-8: Make a program of the following procedure in Problem 5 8: Make a program of the following procedure in probabilistic image processing by using the Gaussian graphical model and the additive white Gaussian noise.
Algorithm: Repeat the following procedure until convergence
1T
22
2
11||1
111Tr
||1
g
CICg
CIC
ttVttt
Vt
T22
242
2
2
11
11||
111
1Tr||
1 gCI
CgCI
I
tt
ttVtt
tV
t
gCI
Im 2)()()(
ttt
2)(minarg)(ˆ tmztf ii
zi
i
K.Tanaka: Introduction of Image Processing by Probabilistic Models, Morikita Publishing Co., Ltd., 2006 .
物理フラクチュオマティクス論(東北大) 52
Sample Program: http://www.morikita.co.jp/soft/84661/g
Recommended