Robust Super-Resolution

Presented By:Sina Farsiu

Project Goals

Understanding & simulation of Dr. Assaf Zomet, et.al. paper : “Robust Super-Resolution” [1]

Comparing the results obtained by this method to other methods

Enhancing the method introduced in: [1]

Super-Resolution Objective

Generation of a high-resolution image from multiple low-resolution moving frames of a scene.

Super-Resolution Formulation

nHXb

pk1 kkkkk nXFCDb

pppp n

n

X

FCD

FCD

b

b

11111

Solutions for S-R Problem

No Noise:

With Noise

bHX 1

, min2

2ILLxHXb

X

bHIHHX TT 1)(

Problem with These Solutions

In the presence of outliers (error in motion estimation, inaccurate blur model, pepper & salt noise, …), these methods do not work accurately. Robust S-R methods can help in these situations.

Robust S-R Formulation

pk1 kkkkk nXFCDb

2

1)(

2

21

p

kkkkk XFCDbXL

)( kkkkTk

Tk

Tkk bXFCDDCFB

Robust S-R Formulation

Robustness

p

kkBXL

1

)(

)(*1 XLXX nn

p1kk y)}(x,median{By)L(X)(x,

987654321 B B B B B B B B B median

Why Median?

Median is an estimate of mean.Unlike regularization method only one of low resolution frames contributes to reconstruct each pixel in the high-resolution frame. So outliers in other frames are discarded in the reconstruction process.Claim: In the absence of additive noise to all frames median method works better than regularization method.

What if noise is added to all frames?

Claim: If considerable amount of additive noise is present in all frames regularization method works as good or even better than median method.

Median-Average Reconstruction

Instead of using

We can combine average and median operators to get better results.

p1kk y)}(x,median{By)L(X)(x,

1

987654321 B B B B B B B B B average

Bias Detection Procedure

It is useful to detect the outliers in the low resolution frames.We can omit those outlier pixels in our procedures. The difficulty is to differentiate between aliasing and outlier effects.

Bias Detection

Formulation

After thresholding non zero values are due to aliasing or outliers.

jkkkk

jk XFCDye

Bias Detection Result

Due to outlier Due to aliasing

1 1 1 1 1

1 1 1 1 1

1 1 1 1 1

1 1 1 1 1

0 0 0 1 1-

0 1- 1 1- 1

0 1 1- 0 0

0 1- 1 0 0

Bias detection Procedure

1: compute 2: Threshold3: Filter the result with a LPF4: Threshold5: Omit corresponding pixels from super-resolution procedure

jkkkk

jk XFCDye

Bias Detection

B-D method works only for uniform gray level outliers.In many situations median operation in robust super-resolution eliminates the bias of the estimator.

Original L-R Frame

Robust S-R Reconstructed H-R Frame mse=0.0017

Median Reconstruction after adding noise 0.0131

Regularized S-R Reconstructed H-R Frame mse=0.0131`

Regularized Reconstruction after adding noise mse=.0125

Error Due to Outlier

Error Due to Aliasing

Conclusion

Robust super-resolution method is quite effective in the presence of outliers, and produces better results in comparison with regularization method. In the presence of additive noise in all low –resolution frames this method loses its superiority to the regularization method.

Conclusion & Results

Combination of mean and median operators can help us in this situation.Proposed bias detection algorithm is an effective method to detect outliers. If outliers are the only source of error in the L-R frames(no additive noise), more iterations we use smaller mse we will achieve.

Suggestions for Future Research

Combining regularization and robust super-resolution methods.Using bias detection results in regularization method.Using robust super-resolution method in frequency domain.

Acknowledgment

Thanks to Dr.Assaf Zomet, Dr.Michael Elad, Dirk Robinson and Dr. Peyman Milanfar for their valuable advices & suggestions.

references

"Robust Super Resolution",   A. Zomet, A. Rav-Acha and S. Peleg Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR), Hawaii, December 2001."Efficient Super-Resolution and Applications to Mosaics",  A. Zomet and S. Peleg, Proceedings of the International Conference on Pattern Recognition (ICPR), Barcelona, September 2000.

“A Computationally effective Image super-resolution Algorithm”, Nguyen, N., P. Milanfar, G.H. Golub, IEEE Transactions on Image Processing, vol. 10, no. 4, pp. 573-583, April 2001 “A Fast Super-Resolution Reconstruction Algorithm for Pure Transnational Motion and Common Space Invariant Blur”, M. Elad and Y. Hel-Or, the IEEE Trans. on Image Processing, Vol.10, no.8, pp.1187-93, August 2001.

Thank You All

Additional Simulatins

Original H-R Frame

Blured median

Projected L-R frame

L-R Frame

Regularization with high noisemse=.0481

Median with high noisemse=.0693

Composite Median & AverageMSE=0.0592

Original H.R. Frame

L.R. Frame Before Adding Noise

L.R. Frame After Adding Noise

Regularized ReconstructionMSE=.0216

Median ReconstructionMSE=.0077