Enhancement of Jpeg Coded Images by Adaptive Spatial Filteirng

8/18/2019 Enhancement of Jpeg Coded Images by Adaptive Spatial Filteirng

1/4

ENHANCEMENT OF JP EG CODED IMAGES BY ADAPTIVE SPATIAL

FILTERING

Amlan

Kundu

US WEST

Advanced Tech.,

Boulder , CO

8 3 3

email

: a k u nd

u@adv t ch

.

uswest oni

A B S T R A C T

The JPEG coder has proven to be extremely useful

in coding image da ta . For low bit -rate image coding

0.75 bit or less per pixel

),

however, the block effect be-

comes very annoying. Th e edges also display ‘wave-like’

appearance. In this paper, an enhancement algorithm is

proposed to enhance the subjective quality of the recon-

structe d images. First, t he pixels of the coded image are

classified into three broad categories [l]:

a)

pixels belonging

to Quasi-constant regions where the pixel intensity values

vary slowly, b) pixels belonging to dominant-edge (DE) re-

gions which are characterized by few sharp and dominant

edges and c) pixels belonging to textured regions which

are characterized by many small edges and thin-line sig-

nals. An adaptive mixture of some well-known spat ial filters

which uses th e pixel labeling information for its adaptatio n

is used ns the adaptive optimal spatial filter for image en-

hancement. Some experimental results are also provided t o

demonstrate th e success of t he proposed scheme.

1. I N T R O D U C T I O N

Recently, a number of schemes for post-processing of still

and video coded images are proposed [1, 2,

31.

The need

for such post-processing arises from the annoying visual ar-

tifacts present in low bit -rate image coding. For instance,

in low bit-rate JPEG still coding, the block effect over the

quuasi-constant regions in the image are very annoying. Th e

edges, on t he o ther ha nd, display ‘wave-like’ appearance. In

low bit-rate vector quantized images, similar block effects

are seen while th e edges look jagged after reconstruction

[4]. The block effects are also prominent in low bit-rate

fractal coding. The post-processing algorithms are essen-

tially smoothing techniques which at tem pt t o remove these

artifacts for bet ter visual perception. Since ther e is no well-

defined mathematical representation for visual perception,

the post-processing algorithms are best designed by proper

understanding of the ad-hsc natu re of the probleni. For th e

problem at hand, i.e., artifacts present in low bit-rate JPE G

coding, we need considerable smoothing t o remove block ef-

fects. On the other hand, t he block effects are somewhat

muted over the tex tural regions. The edges require special

filtering tha t preserves the edges. Besides, the degradation

of edges are spatially somewhat broadened after reconstruc-

tion. Thus, we need to treat t he edgels, texels and the

pixels belonging to quasi-constant regions differently. In

the scheme described in this paper, the pixels in the coded

image are labeled with three distinct labels. This pixel la-

beling is the n used t o ada pt different type of spatial filtering

schemes to different type of pixels.

2 THREE-WAY

P I X E L L A B E L I N G

Our objective is to classify each pixel

as a

member of one

of the three kind of regions ~ the DE regions, the textured

regions, an d the QC regions but not t o explicitly assemble

the pixels into regions.

Edg e1 Det ect io n: First, the edgels are detected. In-

stead of well-known bu t computational ly expensive opera-

tors, we have used the edge operator described in

[5 ]as

this

operato r can be easily tuned to detect the strong edges with

very little computation . The edge operator is defined by

edge-map =

H

o T o R inputimage). (1)

The operator R, for each sliding window, computes a

measure of the range of pixels inside th e window. If this

range is bigger than a threshold

7 1

the cen tral pixel of the

window

is

considered likely to be an edgel. The operator T

then divides th e pixels centered around this likely edgel into

two groups groups of higher intensity and lower intensity

values

).

This stage is controlled by another threshold

7 2 .

Let

N

be the number of pixels belonging to the group of

higher intensity values. Under some mild assumptions, it

can be shown that

N

follows stand ard normal distribution.

The operator H uses a hypotheses testing paradigm t o make

the final determination whether or not the central pixel is

an edgel

[ 5 ]

Texel

Det ect ion : The pixels belonging to the

QC

and

the textural regions are separated from each other using a

simple measurement - the number

of

zero-crossings - and a

thresholding operation. The number of zero crossings are

calculated as follows: In an nxn sliding window, the mean

is calculated over all the pixels and then subtracted from

each pixel. Let the intensi ty value of such a pixel be Ti. As

a

result, some elements will be positive, and some elements

will be negative. Th e change of sign in th e intensi ty val-

ues of two consecutive pixels in any particular direction is

defined

as

a zero-crossing in tha t direction. The number of

zero-crossings NZC ) are comp uted along each row,

along

each column and along the two diagonal directions. Both

the Q C an d the textural regions have

a

rather large value

for the NZC parameter. So, the objective

is

to use the NZC

187

0-8186-7310-9/95$4.00 0 995 IEEE


2/4

parameter along with some LSD (local standar d deviation)

measure to discriminate between the QC a nd th e textural

regions. In our work, a simple parameter reflecting LSD

is

proposed. After the subtraction of local mean, if the ab-

solute value of any pixel inside the window

is

less than

a

preselected threshold value

P ) ,

that pixel value is set to

zero. Otherwise, the pixel value remains unchanged. The

NZC parameter

is

recomputed after t his thresholding oper-

ation . In a QC region, this new NZC value is much less than

it s original NZC value if 3 is properly chosen. In a textural

region, on the other hand, this NZC value is only slightly

less than its original value. Consequently, thi s new recom-

puted NZC (RNZC) parameter could discriminate between

the QC and the textural regions.

This formalization for finding the texels is similar to a-

level crossing problem in stochast ic process theory. Since

the zero crossings, or rather the P-crossings, are counted

along 1-D, we postulate that the intensity values

Ti

along

1-D is the realization

of

a stochastic process where each

random variable has a zero mean. We further assume that

the process is normal an d differentiable. Then th e level-

crossing density (for level

a

is given by

where

R T )

s the autocorrelation of the process. Our ob-

jective is to find a (or p such that , under mild assumptions,

the number of level crossings for textural regions is guaran-

teed t o be much higher th an t ha t of quasi-constant regions.

Let

Ri .r)

be the autocorrelation of the process when the

region under consideration is a QC region. Similarly, let

R ~ T )e th e autocorrelation of the process when t he region

under consideration is

a

textur al region. We assume th at

R l r ) has the form clfi (7) where

f i 0 ) = 1, f i 00) = 0

and

f l / ~ )

=

-vue.

We further assume that

R ~ T )

as

the form c ~ ~L ( . ~ ) c o s ( w T )here fz 0 ) = 1,

Fz 00) =

0 and

f z / - r ) = -ve. To be valid autocorrelation functions, fi .)

and

fi .)

approach zero with

7 4 00

The periodic com-

ponent of R ~ T )s a simplistic assumption but understates

the textural characteristics quite well because the autocor-

relation in a textural region does not

fall

off monotonically

as

it does in a QC region. Let us define

A [texture

X,lQC

region

A =

3)

It can be shown tha t a evel crossings of th e textu ral

region will be much more when the following constraint is

satisfied.

p = a2 > 3c1/2

4)

Here, c1 is essentially the variance of the QC region. If we

assume that the variance of QC regions never exceeds 4%

of th e dynamic range, then for the dynamic range of 0-255,

,

is approximately found to b e 12. To summarize, the pixel

classification algorithm is executed as follows:

The edge operator is first used to locate th e dominant

edges.

If the

R N Z C

is greater than

q

(see Eqn. (5)) at

any of the rem ai ni ng pixel positions, that pixel is

defined as a texel. Otherwise, th e pixel belongs to a

QC region.

q is calculated from the following consideration. Over a

QC region, each pixel value, by assumption, follows a Gaus-

sian distribution

N 0 ,

. Since /3 is approximately equal to

1.220, the probability th at the pixel is not set to zero is ap-

proximately 0.125. For each nonzero pixel, the maximum

contribution to

R N Z C

is

4

ns

this pixel could be counted

4

times ns part four directional countings horizontal, ver-

tical and two diagonal

) . So,

a good first approximation

value of q is

(5)

= n2

4.0.125

Here, n2 is the window size. For 5x5 windows used in

our experiments, an

q

in the range 10

2 )

is adequate.

3

O P T I M A L ADAPTIVE SPA T I A L FI L T E R

F O R

E N H A N C E M E N T

Over the QC regions, the Hodges-Lehman

D

filter is used

for smoothing th e reconstructed image

ns

this filter is known

to b e very efficient [6] in smoothing o ut noise with

a

short-

tailed dis tribution. The coder noise over QC blocks usually

appears as a short-tailed distribution.

3 1

Hodges-Lehman

D

Filter:

Let Xi

1

5 5 n be

a

sample from a population with distri-

bution

F x , ’)

and density f x,

’)

where f .) is symmetric

about zero, continuous and strictly positive on the convex

support of F , [x

0

< F x ) 5

11.

Denote X l ) ,

.,

X n )

as

the order statistics of the sample. Let n=2m or 2m-1. In

either case, we define the

D

filter outpu t

as:

D n x l ,

.,2 =

median

l < i s m X i )

+ X n - i + ~ ) ) .

12.

6)

Thresholded

D filter: First, the pixels with intensity

values in the range (c

Range)

and (c

+

Range)

are

selected. Then, the

D

filter is applied only to the pixels

within this range. Usually, c is the intensity value of the

central pixel in th e window.

Over a DE region, the noise profile appears t o be a long-

tailed di stribution. The median filter is known for its effi-

ciency in smoothing out such noise distribution. Also,

the

me dia n

filter

preserves t he edges Our experimental

findings have shown that, for regions around the dominant

edges, a

5x5

or a 7x7 median filter is generally required for

smoothing. Another filter we have considered is the Multi-

stag e Median Filtering. Consider a nxn window with four

1-pixel wide subwindows along horizontal, vertical and two

diagonal directions. Let these windows be designated as

W I ,

.,W4 Let

zi

be equal to

median

all

pixels

in

Wi

).

The maximum and the minimum of these four median out-

put are computed as q m a Z ) n )nd

y ~ ( ~ ~ ~ ) ( n ) .

et a n ) be

th e central pixel. Then th e outp ut of the multistage median

filter

is

defined as:

Next, we describe the most effective filtering combinations

as determined by t he experiments.

188


3/4

3 2

1.

2.

Adapt ive Fi l te r Combina t ion

A 5x5

or

7x7 median filtering is used to smooth the

edge points where the preceding and the succeeding

points in a raster scan sense

)

of any edge point

are also considered

a s

edge points. This ‘flattening’

of edge can be easily realized by

a

morphological di-

lation operation using a horizontal line element with

thre e elements. Th e filtering is followed by one or two

passes of 3x3

D

filtering over th e QC regions. F ilter-

ing over the textural regions shows no visual improve-

ment. It should be noted tha t t he texels could easily

appear as ‘too much smoothed’.

One pass of 3x3

D

filtering over the Q C regions is fol-

lowed by 5x5 multi-stage median filtering t o smooth

the edge points where the preceding a nd th e succeed-

ing points of any edge point ar e also considered

as

edge points . Th e filtering is followed by one pass of

3x3 thresholded

D

filtering over the entire image.

For

color images, the individual R, G and

B

components

are filtered using th e schemes outlined above. The 3-way

pixel classification information

is

obtained from the 3-way

pixel classification of

Y

image where the greyscale image

Y

is defined as Y

=

0.2999. R + 0.587. G + 0.114. B.

The filtering on

Y , U

and V images (as opposed to R, G, B

images) show distu rbing color variations. Such observation

is previously made in

[3].

4 V I S UA L C O M P A R I S O N

TEST

A N D

E X P E R I M E N T S

Since no universally accepted quan titative criterion exists

to express th e visual quality of images,

a

subjective visual

comparison test has been designed.

A

pair of images are

displayed on the screen side by side. Th e two images are

created by two different filtering schemes (one image could

lie the original coded image), and they are combined in

a random fashion. The subjec ts, i.e. human observers,

are instructed to choose one image over the other. If this

experiment is carried out for

a

sufficiently large number of

human subjects, the image with the better visual quality

is likely to be chosen more often than the other; and the

experiment is very likely to reveal which filtering schemes

provide the maximum improvement in visual quality.

The segmentation based enhancement algorithm is tried

on one coded image (0.625 bit/pixel) as depicted in Fig.

l(a). Figures l(b) and l(c) show the enhanced images us-

ing the filtering schemes

2)

nd

1)as

described before.

Figure l( d ) shows 3-way pixel classification of Fig. l( a) .

By means of visual comparison test

as

described above, it

is corroborated that the enhancement schemes do improve

the visual quality of the images. This experiment is done

on

a

few other greyscale. images with similar observations.

The visual improvement, particularly in removing blocking

effects, lends itself

to

some form of quantitative representa-

tion

ns

shown below.

4 1 11.11 Measure o Difference mage

Consider the difference image obtained by subtracting the

original image from the coded or the post-processed image.

Let th e difference signal image be s z,

j )

over the QC regions

and t he local average of

s i , j )

be

r z , j ) .

Indicat e the set of

i,

) which belongs to t he QC regions by R. The measures,

M I is now defined

as

follows:

where N is the number of points in R. M 1 measures the

average absolut e deviation of the intensity of the difference

or error image. For ideal filter performance, M1 should be

zero, i.e., the difference image is constant. It is relevant

to note here that the addition of a constant to its pixel

values does not impair the visual quality of the image in

any noticeable way as long as the constant is small com-

pared t o the dynamic range of the pixels. For the coded

image (Fig. l(a)), M I is found to be 1.44 while the post-

processed images using both schemes

1)

and 2) yielded

M I of 1.32. Similar improvement in A41 is found in other

greyscale coded and post-processed images. This measure,

however, is not meaningful for th e texels and th e edgels.

For color images with bright colors, the blocking effect

and the edge degradation are not often visible until the bit

rate is very low (1:30 or even less). The bright colors tend

to conceal the coding artifacts. The same filtering scheme

is found effective on very IOW bit-rate JPEG coding of color

images, but filtering is done separately on individual R, G

and

B

frames as mentioned before.

As JPEG coding

is

gaining wide popularity, this en-

hancement technique will find applications as a value-added

technique particularly for low bit-rate coding situations.

Some preliminary experiments are also performed on low

bit-rate fractal coded greyscale images using similar filter-

ing schemes. Once again, dist inct visual improvement is

noticed in reducing blocking effects.

5

R E F E R E N C E S

R.A. Gopinath

et

al.

,

”Wavelet based post-processing

of low bit rat e transform coded images”, Proc. of ICIP,

Austin, Texas,

pp.

913-917, 1994.

T. -S. Liu and

N.

S. Jay ant , ”Adaptive post-processing

algorithms for low bit rate video signals”, Proc. of

ICASSP ’94, Adelaide, Australia, pp . V-401

-

V-404.

V. Ramamoorthy, ”Removal of staircase effects in

coarsely quant ized video sequences”, Proc. of ICASSP

’92, San Francsisco, CA, pp. 111-309 111-312, 1992.

”Enhancement of Vector Quantized Images by Adap-

tive Nonlinear Filtering,” patent awarded ( 5218649)

to US WEST with

A.

Kundu, V. Ramamoorthy and

M. Terry

as

the authors.

A .

Kundu and S K . Mitra, ”Image Edge Extraction

Using A Statistical Classifier Approach,” IEE E Trans.

on PAMI, PAMI-9, 4, 569-577, 1987.

P. J. Bickel and J. L. Hodges, ”T he Asymptotic The-

ory of Galton’s Test and a Related Simple Estimate of

Location,”

Annals of

Mathematical Statistics,

38,

pp.

73-89, 1967.

189


4/4

Figure

1:

a) Original Image. b) Enhanced image using scheme

2).

Figure 1: c ) Enhanced image using scheme 1).

( d ) 3-way pixel labeling

of

Fig. l a).

190

Documents

Enhancement of Jpeg Coded Images by Adaptive Spatial Filteirng