Adaptive Filtering of Raster Map Images

Adaptive Filtering of Raster Map Images

Minjie Chen*, Mantao Xu and Pasi Fränti

Speech and Image Processing Unit (SIPU)School of Computing

University of Eastern Finland, FINLAND

Raster Map Images

Topographic or road maps Few colors Detailed spatial structures

Filtering of Raster Map Images

RequirementNo over-smooth, remain readableNumber of colors does not increasePreserve spatial structures

Noise ModelImpulsive NoiseGaussian Noise

Original: 4 colorsScanning Image:

11063 colors

Quantized Image:4 colors

Existing Methods

Impulsive NoiseVector Median (VM, AVM)Peer Group Filtering (PGF,FPGF )Morphological FilteringContext Tree ModellingDiscrete Universal Denoiser (DUDE)

Gaussian NoiseWavelet denoising using Gaussian scale mixtures (GSM)Non-local mean (NLM)Dictionary- based method (K-SVD)Blocking matching and 3D filtering (BM3D)Markov random fields/conditional random fields (FoE,ARF)Non-Local Sparse Models for Image Restoration (NLSM)Patch-based Locally Optimal Wiener Filtering for Image Denoising (PLOW)Sparsity-based Image Denoising via Dictionary Learning and Structural Clustering (CSR)

Most algorithms are designed for continuous-tone(photographic) imagesComplicated spatial structures in the map will be destroyed

Multi-Layer Method for Morphological Filtering

Covert multi-dimension image filtering problem into a series of binary image filtering problem using layer seperatingAfter filtering on each layer, select suitable layer ordering to reconstruct the image

Step 1: Divide color image into Multi-Layer binary imagesStep 2: Filter binary layers separatelyStep 3: Layer ordering decisionStep 4: Merge the filtered layers

Layer separating

Filter each layer

Merging step with global color priority

Example of Multi-Layer Method(Global Color Priority)

Step 3: Layer ordering (cont.)

Global criterion : select colors according to their frequency

Lowest priority is used as background color

PRIORITY

Highest

Priority

618416 230020 173358 22975 3458 349

Same color priority for whole imageNo difference for different regions

Step 3: Layer ordering: Local Color Priority (Chen et al. ICIP’09)

Segment the image into several regions with different background color, set different ordering criterions for these regions

Merging with local color priority

Example of Multi-Layer Method(Local Color Priority)

Algorithm for color priority decisionStep 3.1: Dilation and fill holes

Process each color layer by soft morphological dilation and filling holes operations.Large blocks regions are possible background regions.

After dilation and filling holes

Algorithm for color priority decisionStep 3.2: Evaluate preliminary segments

Region labeling labels connected pixelsAfter region labeling, sort segments according to their sizeIf the size of the segments is larger than a threshold, select as background region candidateThe goal is to extract large connected segments in each layer for later background filling step

Example of Step 3.2Preliminary segments

Segment Size

1 765241

2 662515

3 187677

4 83904

5 51170

6 21044

Six large segments are detected from different layer

2,5 from black layer, 3 from blue, 1,4 from white, 6 from brown

Algorithm for color priority decisionStep 3.3 Background filling

Evaluate if those large segments detected in Step 3.2 are real background regions.First create a blank background imageEvaluate those segments one by one from large segment to small segment.If it is real background, add to background image, labeling the region with the layer color

Algorithm for color priority decisionStep 3.3 Background filling

Two features used for validating if it is background segmentFeature 1: How many change after dilation and image filling operationBackground segment’s size does not change much, have large ratio

Region 1: Before 526337 After 765241

Ratio 0.6878(ADDED)

Region 2: Before 193606 After 662515

Ratio 0.2922(NOT ADDED)

Algorithm for color priority decisionStep 3.3 Background filling (cont.)

Feature 2: Does this segment overlap with background already filled?In common condition, it should have a small value, this feature can enhance the filling accuracyIf feature 1 is close to 1, this feature can also be large, it will add a background inside one background segment

Region 2: Overlap percentage 97% (NOT ADDED)

Region 1: Overlap percentage 0% (ADDED)

Already labeled region 1

Blank Background

Fill block 1

S1/S2 Po ADD(Y/N)

1 0.6878 0 Y

2 0.2922 0.9712 N

3 0.8373 0.0172 Y

4 0.8024 0.0149 Y

5 0.2302 0.9953 N

6 0.1567 1 N

Fill block 3

Fill block 4

Blank background

image

We can set different threshold in filling step, causing different background images.Classification can be done to decide threshold for different type of images.

Algorithm for color priority decisionStep 3.4 Process unlabeled region

Large unfilled regions ---- set as small background region.Small unfilled region ---- merge it to the nearest background.

Set as small background region

Merge to the nearest background

After processing

Step 3.5 Calculate the color priority in different background regions

For all background region, calculate its corresponding color histogram We then get different color priority for different region

More examples of background image

Result - impulsive noise

Conclusion of Multi-layer method

The current method does not work if color number is largePriority between small structures not consideredSpecial local patterns combined with multiple color is missed

Statistical Filtering (Chen et al. ICME’10, IEEE TMM’ 11)

Colors with low conditional probability will be replaced by the dominated color

Frequency Frequency

X X

Frequency

4093459

Frequency

22

19336

Context Template

Example of Context Tree Modeling

Context Dilution 6-color, 20-pixel template has 620 = 3,656,158,440,062,976 contexts

Most contexts has rare appearance, cause inaccurate conditional probability estimation.

Context Tree ModelingOnly appeared contexts are allocated in memory

Time complexity O(N), where N is the length of a data sequence.

Tree spanning is terminated once the frequency of the context on a given node is less than a predefined

value(N) Threshold Contexts

N=256 23676

N=128 35847

N=64 54736

N=32 82260

X XX X1 1 1

Root

1N1=220N2=110N3=25N4=35

N1=405N2=117N3=238N4=6

N1=46N2=810N3=44N4=75

N1=400N1=58N1=24N1=30

Children pointer array

X1

2 X1

2 X1

2N1=23N2=67N3=25N4=5

N1=85N2=16N3=0N4=10

N1=112N2=27N3=0N4=20

Context Merging

Including noise pixels in the surrounding contexts makes good conditional probability estimation difficult.

For contexts with rare appearance, a merging process is done to collect the statistics of all similar contexts.

Time complexity is O(kNM2), where M is the number of colors, k is the depth of the context tree, N is the number of contexts with rare appearance.

Example of context merging

Noisy pixelMost consistent sub-context

Filtering Threshold

Discrete Universal Denoiser(DUDE)

M is the number of colors, δ is the noise level, u0 is the color with highest conditional

probability.

DUDE has a so-called “asymptotic optimality” property for M-ary symmetric noise.

Estimation of δ

Estimated by the minimum conditional probability occurred for contexts with “sufficient

frequency”:

0 arg max ( | )x Au P x c

2

0 0( 1) (1 ) ( 1)( | ) ( | ) 1

((1 ) 1) ((1 ) 1)M MP x x P x u

M M

c c

2, ( ) 101 max ( | )

c

cx p

p x

0 0( ( ) | ) / ( 1), 1,2,...,P I x x M x M cThis decision rule is designed for the count statistics collected on the noisy image. For a clean image, the decision rule is

Extension for Gaussian Noise

Iterative algorithm to optimize both the estimation of the indexed image and its color palette.

The distance between RGB color vector to its corresponding component in the color palette, and its conditional probability of local context are taken into account as an information fusion. Definitions:

X: index image, Y: corresponding image, CP=(m1,m2,…mM) color palette where mi = (mi(r), mi(g), mi(b)) , yx = (rx,gx,bx) the color intensity of x in RGB space.

Input: YX, CP ← Conduct color quantization based on Y Σ ← Estimate the quantization variance based on

X, Y,CP according to (3)For T iterations DO: Given X, update P(x|c) Update X , according to (1) Update CP and σ according to (2) and (3)End-ForOutput: Y, X, CP.

General Scheme for filtering Gaussian noise

(1.. ) 2 2

2

2

2

( ) arg min ( log ( | ) log ( | ))

~ ( , ), is the mean variance

|| || ( ) exp( )

2

x M x

x

I x f P x

where N

f

x

x

x xx x

y M c

M m I

y my | M

( ) median({ | , and ( ) }) x Xim r r x I x i

2

1 , ( )

1 || ||3 | |

M

i x I x iX

x iX

y m

(1)

(2)

(3)

Filtering Example

Noisy image Quantized image

Filtering after 1 iteration Filtering after 5 iterations

Experiments under Different Noise Level

δ = 0.05

Error rate (%)

AVM PGF CT DUDE ACSSet#1 8.65 1.40 1.04 0.94 0.65Set#2 4.19 1.26 0.99 0.99 0.78Set#3 11.8 3.26 2.41 2.09 1.59Set#4 6.55 1.21 1.42 1.34 0.82Set#5 9.27 6.02 4.36 2.66 2.02

σ =25 PSNR

GSM NLM BM3D ARF ACSSet#1 24.4 24.9 26.2 24.2 47.1Set#2 24.0 23.5 26.5 23.3 59.2Set#3 23.7 22.4 23.7 23.0 27.0Set#4 24.6 24.1 25.5 23.8 33.4Set#5 25.1 24.3 26.1 25.1 27.7

0 5 10 15 200

1

2

3

4

5

Impulsive Noise Level(%)

Erro

r Rat

e(%

)

AVMPGFCTDUDEACS

0 10 20 30 40 50

20

25

30

35

40

45

50

Gaussian Noise()

PS

NR

BM3DNLMGSMARFACS

Visual Examples

Impulsive noise AVM PGF CT DUDE ACS

Gaussian noise BLS-GSM NLM BM3D ARF ACS

Mixed noise PGF BM3D BM3D+PGF FPGA ACS

Summary

Statistical filtering for raster map images

Can process images distorted by impulsive noise, additive Gaussian noise and mixture Gaussian-impulsive noise

Both color distribution in RGB space and the conditional probabilities of local context are considered

Optimize the Context Selection (Chen et al. ICIP’11)

Pruning? Merging?

Reorder the context pixel-wise?

Our solution: An voting-based method to optimize the context selection

Measuring Goodness Score of Context

identify good or bad context by top-down tracing from the root of tree.

|2 |

( | )( )F( ) log ( ) k ( | ) log( )( ) ( )

X CX C

xE

P xP P xP P x

ccc c

c

1

( ) ( )

k

E ii

P P yc

( ) is probability of context , ( ) is the probability of color i iP P y yc c

Example of pixel with good Context and bad Context in the Map

Original NoisyOriginal Noisy

Noisy pixel with bad context

Noisy pixel with bad context can not filter correctly.Can we use bad context for noise estimation?

Detect Noisy Pixel by bad Context

Bad contexts will have higher score difference after the

noisy pixel is removed.

We can accumulated these score difference for bad context to find

noisy pixel, which is the voting image R:

Include noisy pixel may cause bad context.

/{ }, 1..R( ) = R( ) + F( ) - F( ) when F( ) - F( ) 0

is the pixel position of th context element

i i

i i i

i

i

x i kx x

x i

z cz c

z c

Voting Image (cont.)

F(z)- F(c)-0.39 2.29 5.41 -0.38

0 0 0 0 0

0 0 0 0 0

0 0 0 0 0

0 0 0 0 0

0 0 0 0 0

Voting image

0 0 0 0 0

0 0 0 2.29 0

0 0 5.41 0 0

0 0 0 0 0

0 0 0 0 0

0 0 0 0 0

0 0 0 2.29 0

0 0 5.41 0 0

0 0 0 0 0

0 0 0 0 0

+

Voting Image (cont.)

F(z)- F(c)1.39 -1.24 -1.33 3.58

0 0 0 0 0

0 0 0 2.29 0

0 0 5.41 0 0

0 0 0 0 0

0 0 0 0 0

Voting image

0 0 0 1.39 0

0 0 3.58 2.29 0

0 0 5.41 0 0

0 0 0 0 0

0 0 0 0 0

0 0 0 1.39 0

0 0 3.58 0 0

0 0 0 0 0

0 0 0 0 0

0 0 0 0 0

+

Voting Image (cont.)

F(z)- F(c)-0.62 5.69 1.96 -0.40

0 0 0 1.39 0

0 0 3.58 2.29 0

0 0 5.41 0 0

0 0 0 0 0

0 0 0 0 0

Voting image

0 0 0 1.39 0

0 0 3.58 2.29 0

0 0 11.0 0 0

0 1.96 0 0 0

0 0 0 0 0

0 0 0 0 0

0 0 0 0 0

0 0 5.69 0 0

0 1.96 0 0 0

0 0 0 0 0

+

Voting Image (cont.)

F(z)- F(c)4.15 -2.86 -2.32 2.24

0 0 0 1.39 0

0 0 3.58 2.29 0

0 0 11.0 0 0

0 1.96 0 0 0

0 0 0 0 0

Voting image

0 0 0 1.39 0

0 0 7.73 2.29 0

0 2.24 11.0 0 0

0 1.96 0 0 0

0 0 0 0 0

0 0 0 0 0

0 0 4.15 0 0

0 2.24 0 0 0

0 0 0 0 0

0 0 0 0 0

+

-0.65

Voting Image (cont.)

F(z)- F(c)-2.02 3.23 -1.40 -1.63

0 0 0 1.39 0

0 0 7.73 2.29 0

0 2.24 11.0 0 0

0 1.96 0 0 0

0 0 0 0 0

Voting image

0 0 0 1.39 0

0 0 7.73 2.29 0

0 2.24 16.5 0 0

0 1.96 3.23 0 0

0 0 0 0 0

0 0 0 0 0

0 0 0 0 0

0 0 5.51 0 0

0 0 3.23 0 0

0 0 0 0 0

+

5.51

Voting Image (cont.)

F(z)- F(c)1.79 1.38 -1.43 -1.16

0 0 0 1.39 0

0 0 7.73 2.29 0

0 2.24 16.5 0 0

0 1.96 3.23 0 0

0 0 0 0 0

Voting image

0 0 0 1.39 0

0 0 7.73 2.29 0

0 2.24 18.3 2.35 0

0 1.96 3.23 1.38 0

0 0 0 0 0

0 0 0 0 0

0 0 0 0 0

0 0 1.79 2.35 0

0 0 0 1.38 0

0 0 0 0 0

+

2.35

Noisy pixel have a high voting value

Voting Image (cont.)Bad context include black noisy pixel

south east

reduced context

F(z)- F(c) 4.15

5.69

Voting value7.73

Bad context include white noisy pixel

north west south west

south

F(z)- F(c)

reduced context

5.41 5.51 1.79Voting value

18.30

3.58

Voting Image (cont.)

voting image

NoisyOriginal

Voting Image (cont.)

voting image noisy image

Adaptive Context Selection

If contexts are not good context, pixels with low voting value are selected to construct new adaptive context.

Filtering Result for Impulsive Noise

Original Noisy

Iteration 1 Iteration 2

DUDE Proposed

Noisy pixels with contaminated contexts are filtered correctly by optimal context selection

Conclusions

Adaptive context selection via a voting-based noise estimation scheme

Can process raster map images distorted by impulsive noise, additive Gaussian noise or mixture Gaussian-impulsive noise

Extension for optimizing the context selection for denoising gray-scale image, e.g. voting-based method to optimize the weighting coefficient in NLM, PLOW, K-SVD.

Related Paper

M. Chen, M. Xu and P. Fränti, "Multi-layer filtering approach for map images", IEEE Int. Conf. on Image Processing (ICIP'09), Cairo, Egypt, 3953-3956, 2009.M. Chen, M. Xu and P. Fränti, "Statistical filtering of raster map images", IEEE Int. Conf. on Multimedia & Expo (ICME'10), Singapore, 394-399, 2010. (oral)M. Chen, M. Xu, P. Fränti, "Adaptive Context-tree based Statistical Filtering of Raster Map Images Denoising", IEEE Trans. on Multimedia, 16(3), 1195-1207, 2011.M. Chen, M. Xu, P. Fränti, "Adaptive Filtering of Raster Map Images Using Optimal Context Selection", IEEE Int. Conf. on Image Processing (ICIP’11), 77-80, Brussels, Belgium, 2011.(oral)