Introduction to Astronomical Image Processing

8. Mid-level processing(data preparation)

André JalobeanuLSIIT / MIV / PASEO group

Jan. 2006

lsiit-miv.u-strasbg.fr/paseo

Master ISTI / PARI / IV

PASEO

Mid-level processing:data preparation

Data presentationEnhanced visualizationColor display of multidimensional dataMultiresolution vision model

Dimensionality reductionDimensionality reduction methods, Band fusion

Object extraction and signal/noise separationSNR maximization via binningSignificant information detection

Combining multiple observations: data fusionFrame co-addition, mosaicingSuper-resolution, multiframe restoration

Data presentation

Learn how to better display grayscale or RGB color images

Get acquainted to problems related to multispectral image visualization

See how nonlinear transforms and multiscale representations can help visualize lots of information at once

Enhanced visualizationsingle band (grayscale) images

High dynamic range, difficult to visualize! linear pixelwise transforms (e.g. histogram clipping)! nonlinear pixelwise transforms (e.g. log-scale)! multiple pixel operations (e.g. opacity filtering, unsharp mask)! grayscale to RGB mapping: false color display

his

togra

m c

lippin

g

M51 in f

als

e co

lor

(C. Buil)

Enhanced visualizationsingle band (grayscale) images

Opacity masking (M51)

Unsharp masking (Sun spot)

X+k.(X-X*g)

X.f(X*g)

amplify details(image - blurred image)

reduce high pixelintensities,opacity mask=blurred image

C.

Buil

C.

Buil

g = Gaussian blur

Enhanced visualization3-band (RGB) images

๏ Color space manipulations (e.g. HSV space)

‣ Saturation adjustment (S)Compensate for overlapping filters (poor color image)‣ Color cycling (H)Try different colors for a better visual segmentation‣ Nonlinear intensity transform with color preservation (V)Useful for high dynamic range images

Before and after saturation adjustment

C.

Buil

Saturation-enhanced “true colors” (400, 560, 910nm)

HSV space

C.

Buil

Color display of multidimensional data

๏ Multiband image reduction & display [Petremand 04]

‣ PCA or ICA reduction (keep at least 6 bands)‣ Markovian segmentation: label map‣ Discriminant Factor Analysis: color (H,S)Work in the HSV (Hue Saturation Value) space‣ PCA for each class: V‣ HSV to RGB conversion

6 reduced bands (from 48) Label map and color display

High dimensionality of multi/hyperspectral images:Impossible to visualize them as they are!

Reduction to 3 bands (RGB) using PCA, ICA... insufficient (stationary model)

Multiscale Vision Model

๏ Use a multiresolution transform (wavelets)

‣ Select significant coefficients (above noise)‣ Grow regions defined by the significant coefficient location‣ Keep regions according to the inter-scale, region-based connectivity(image segmentation)

[Bijaoui 95]

Galaxy MVM segmentation Connectivity

Dimensionality reduction

Grasp the ideas behind classical dimensionality reduction methods

Understand why such methods are not well-suited to astronomy

Get to know recent techniques used to merge multispectral data into a single band image

Dimensionality reduction methods

๏ Principal Component Analysis (PCA) methodsMaximum variance subspace, Gaussian assumptions

‣ Simple PCA - Gaussian cluster, no noise, one step‣ Probabilistic PCA, Factor Analysis - add some noise (indep.)‣ Mixture of PCA - mixture of PCA models: multiple clusters, nonlinearity

๏ Independent Component Analysis (ICA)Non-Gaussian components assumption

‣ Simple ICA - maximize the nongaussianity‣ Probabilistic ICA - add some noise‣ Projection pursuit - more general: maximize projection index‣ Mixture of ICA - multiple clusters, nonlinearity

linearembedding

subspace

Linear vs. nonlinear embedding subspace

iterative algorithms,

EM used in probabilistic

approaches

Major drawbacks:! ◆ Stationarity assumption! ◆ No image formation model (e.g. blur)

linearembedding

subspace

๏ Problems specific to astronomical images‣ Spatially adaptive behavior‣ Dark background & photon noise‣ Sources multiplied by spatially variable density (e.g. nebulae)‣ Compound sources (e.g. star clusters, galaxies)‣ Complex sources (e.g. stars)

๏ Problems related to Integral Field Spectroscopy‣ Very large number of bands (thousands)

One spectrum for each spaxel, contains lots of informationthat needs to be preserved!‣ Very small number of spatial samples (until Muse, 2012)

Dimensionality reduction of deep field images

Synthetic image

2-band histogram

Color histogram(Hue,Saturation)

Better goal: Source separation or decorrelation

! Find minimum number of independent sources! ! To preserve information, may be larger than the number of bands!

! Determine their spatial distribution (spatial consistency)

Band fusion

๏ Merge multidimensional data into a single bandSNR maximization & detection purposes

‣ Simple averaging (naive, inaccurate)‣ Weighted averagingOptimal averaging, stationary noise‣ Spatially adaptive averaging

•Use image space information(e.g. saturation, bad pixels)

•Use a multiscale approach based onwavelet transforms

Wavelet domain band fusion using Hidden Markov Trees(combined with Van Cittert deconvolution) [Flitti 05]

Object extraction and signal/noise separation

Grasp the principles of SNR enhancement using spatial grouping

Become familiar with the significant information extraction problem and the related challenges

SNR maximization via spatial binning

๏ Uniform binning

‣ Sum or average pixel values withinpredefined super-pixels

๏ Spatially adaptive binning

‣ Find image space subdivision such thatSNR=constant (prescribed SNR)

•Quadtree plane partition

•Voronoi tessellation

X-ray data binning, by S. Diehl & T. StatlerVoronoi tessellation

Maximize the SNR by averaging samples(Central Limit Theorem: σ →σ/√n)

Quadtree partition

Significant information detection

๏ Nonparametric approaches

‣ Strong denoising (ensure a minimum false alarm rate)Use existing methods and minimize the residual noise‣ Enhanced visualization of significant transform coefficients(e.g. Multiscale Vision Model)

๏ Parametric methods

‣ Supervised object detection

•Use template matching to detect predefined patterns

•Recursive object detection/subtraction (e.g. CLEAN)

•Feature extraction (e.g. local maxima)

•Statistical fitting (parametric objects), decision theory‣ Unsupervised object extraction & optimal representation

•Pixon approach [Pina-Puetter 93]:Set of parametric shapes (dictionary), simplest representation→ Keep the parameters (shape, scale, location)

•Fully Bayesian approaches, to be developed

•Both spatial and spectral methods (for IFS), to be developed

Combining multiple observations: data fusion

Understand how to combine multiple images into a single one

See how super-resolution can help preserve information and extrapolate a limited bandwidth

Get acquainted to some techniques used to increase SNR, FOV and bandwidth in astronomy

Data fusion: introduction

๏ Multisource data fusion objectives

‣ Optimally combine all observations into a single imageCo-add or build a mosaic, depending on the overlap‣ Preserve all the information from the original data setIncrease resolution if needed, compute the uncertainties‣ Optional: enhance the image qualityDenoise or deblur depending on the degradation

๏ Possible approaches

‣ Forward methodsShift-and-add, register-and-add, simple resampling schemes, drizzle‣ Inverse methodsBayesian super-resolution, probabilistic fusion, multiframe deconvolution

Problem: lots of data, same object!

Usually, images are recorded with various:! ◆ pose parameters (position, orientation)! ◆ sensors (resolution, noise, bad pixels)! ◆ observing conditions (transparency, seeing)! ◆ telescopes (PSF, distortions)

Frame co-addition in deep-field imagingincrease SNR & dynamic range

๏ Register-and-add

‣ Apply geometric corrections: resample input imagesAvoid simple interpolation schemes!‣ Add resulting imagesUse weighted sum to account for noise variationsApply rank filters to remove cosmic rays & bad pixels

๏ Drizzle

‣ Designed to handle undersampled HST images [Fruchter 96]‣ Shrink input pixels, project onto finer output grid (scale ≈0.5-0.7)Perform geometric corrections, including distortions ‣ Add: Image update iteration (add a drop)

X ← (awY+WX)/(aw+W), W ← aw+W

Y = drop value (pixel value in the input image)a = intersection btw. drop and fine output gridw = user-defined pixel weight (significance)

potential problems:

noise correlation,image blurring,sampling...

Prerequisite: accurate camera calibration(estimate the registration parameters)

X

Y

Image mosaicingincrease the FOV

๏ Same as image co-addition, with some specificities

‣ In all cases:

•Accurate camera calibration using overlapping areas

•Image normalization (flat, transparency, exposure) to avoid boundary effects‣ Register-and-addApply geometric corrections (registration), add images in overlapping areas‣ DrizzleShrink pixels, project (geometric transform), add drops to finer pixels

O’D

ell /

NA

SA

Image mosaicing in planetaryand wide field astronomical imaging

Output pixel size < input PSF FWHM / 2

Super-resolutionde-aliasing and bandwidth extrapolation

๏ Go beyond the sampling resolution - multiple imagesStack of generally undersampled images

‣ Image reconstruction & deconvolution techniques

•3 steps: deconvolution, registration, averaging

•Combined: deconvolution with specificities (multiplicity, sampling):multiple data terms, explicit subsampling, batch or recursive processing ‣ Super-resolution without deblurring (register-and-add, drizzling)

๏ Go beyond the diffraction limit - single imageOne blurred, oversampled image (could be subsampled/binned without loss)

‣ Model-based approachesUse powerful image models based on prior knowledge!Reconstruction possible beyond cutoff frequency if near-black object [Donoho 92]

Super-resolution for point sources using pixons (1 image)

Pixon L

LC

4x super-resolution from 16 images

[Wille

tt 0

4]

Multiframe restorationpartial turbulence compensation

๏ Reconstruction & Deconvolution methodsNegligible phase perturbations: constant PSF, small shifts

‣ Specificities: multiple data terms, batch or recursive processing,shift or undistort (space-variant shifts) before/after deblurring

๏ Handling phase perturbations

‣ Blind deconvolution methods (time-varying PSF)Specificity: constant underlying image‣ Speckle imaging

•power spectrum averaging (lost phase information)

•constrained power spectrum and phase reconstruction‣ Phase diversityRecord in-focus and out-of-focus images simultaneously

Imaging through turbulence: random phase perturbations

Exam

ple

of

image

sequen

ce (

star)

Incremental Poisson MAP restoration.One of the observed images,results with 10 and 50 images[Sheppard 98]

Further reading

๏ Enhanced visualization from the Iris tutorialhttp://www.astrosurf.org/buil/iris/tutorial1/doc2_us.htmhttp://www.astrosurf.org/buil/iris/tutorial5/doc15_us.htm

๏ MultiColorViz (hyperspectral image display)http://lsiit-miv.u-strasbg.fr/paseo/research.php#proj5

๏ Tutorials on dimensionality reductionhttp://www.oid.ucla.edu/Webcast/ipam/

๏ Voronoi tessellation & adaptive binninghttp://www.phy.ohiou.edu/~diehl/WVT/

๏ Dithering and Drizzling (A. Fruchter)http://www-int.stsci.edu/~fruchter/dither/dither.html

๏ Phase reconstruction and phase diversityhttp://www.phy.hw.ac.uk/~phyhic/Theory%20Pages/

๏ Super-Resolution (S. Borman)http://www.seanborman.com/publications/