ACM-LULP

Layered Unequal Loss Protection with

Pre-Interleaving for Fast Progressive Image

Transmission Over Packet-Loss Channels

Jianfei Cai

Nanyang Technological University

Xiangjun Li

Nanyang Technological University

and

Chang Wen Chen

Florida Institute of Technology

Most existing unequal loss protection (ULP) schemes do not consider the minimum quality re-quirement and usually have high computation complexity. In this research, we propose a layeredULP (L-ULP) scheme to solve these problems. In particular, we use the rate-based optimal so-lution with a local search to find the average forward error correction (FEC) allocation and usethe gradient search to find the FEC solution for each layer. Experimental results show that theexecuting time of L-ULP is much faster than the traditional ULP scheme but the average distor-tion is worse. Therefore, we further propose to combine the L-ULP with the pre-interleaving tohave an improved L-ULP (IL-ULP) system. By using the pre-interleaving, we are able to delaythe occurrence of the first unrecoverable loss in the source bitstream and thus improve the lossresilience performance. With the better loss resilience performance in the source bitstream, ourproposed IL-ULP scheme is allowed to have a weaker FEC protection and allocate more bits to thesource coding, which leads to the improvement of the overall performance. Experimental resultsshow that our proposed IL-ULP scheme even outperforms the global optimal result obtained byany traditional ULP scheme while the complexity of IL-ULP is almost the same as L-ULP.

Categories and Subject Descriptors: I.4.2 [Computing Methodologies]: Image Processing andComputer VisionCompression

General Terms: Algorithms

Additional Key Words and Phrases: Progressive image transmission, unequal loss protection, jointsource-channel coding, forward error correction, packet loss

Authors address: Jianfei Cai and Xiangjun Li, Center for Multimedia and Network Technol-ogy, School of Computer Engineering, Nanyang Technological University, Singapore 639798,{asjfcai,pg05304684}@ntu.edu.sg. Chang Wen Chen, Dept. of Electrical and Computer Engi-neering, Florida Institute of Technology, FL 32901, [email protected].

This research is partially supported by Singapore A*STAR SERC Grant (032 101 0006).This paper has been presented in part at the 2004 SPIE Visual Communications and Image Pro-cessing Conference (VCIP04), San Jose, CA, U.S.A., Jan. 2004, and the 2004 IEEE InternationalConference on Image Processing (ICIP04), Singapore.Permission to make digital/hard copy of all or part of this material without fee for personalor classroom use provided that the copies are not made or distributed for profit or commercialadvantage, the ACM copyright/server notice, the title of the publication, and its date appear, andnotice is given that copying is by permission of the ACM, Inc. To copy otherwise, to republish,to post on servers, or to redistribute to lists requires prior specific permission and/or a fee.c 2005 ACM 0000-0000/2005/0000-0001 $5.00

ACM Journal Name, Vol. V, No. N, June 2005, Pages 10??.

2 Jianfei Cai et al.

1. INTRODUCTION

Wavelet-based image codecs, such as SPIHT [Said and Pearlman 1996] and JPEG-2000 [Taubman and Marcellin 2002], have shown to be superior to DCT-basedimage codecs in not only the coding efficiency but also the functionalities such as theprogressive property, which allows the bitstream to be truncated in any position andthus provides rate scalability. Although the progressive image coding can provideexcellent compression and scalability performance, it makes image bitstreams verysensitive to channel noise such as packet-loss in the Internet and bit errors in wirelesslinks. An error in a bitstream may cause all the following bits become useless.Therefore, error control techniques such as forward error correction (FEC) andautomatic repeat request (ARQ) are needed to combat with channel noise to ensurea reliable image transmission.

Recently, we have seen extensive studies in FEC-based joint source channel cod-ing (JSCC) for progressive image and video transmission including [Chande andFarvardin 2000; Nosratinia et al. 2003; Stankovic et al. 2003; Wu et al. 2002] fortransmission over noisy channels and [Mohr et al. 2000; Kim et al. 2003; Grangettoet al. 2002; Stuhlmuller et al. 1999; Stockhammer 2002; Schaar and Radha 2001;Li and et al. 2003] for transmission over packet loss channels. The common idea ofthese schemes is to combine progressive source bitstreams with unequal error/lossprotection (UEP/ULP), i.e., the more important information is given more pro-tection. The progressive image bitstreams can be well fit into UEP/ULP becausethe earlier portions of a progressive image bitstream are always more importantthan the later parts. Comparing with equal error/loss protection (EEP/ELP),UEP/ULP can obtain considerable performance gain and have the property ofgraceful performance degradation during channel mismatch cases while the com-plexity of UEP/ULP is much higher than that of EEP/ELP since it is not trivialto find the optimal UEP/ULP solution.

Many schemes [Chande and Farvardin 2000; Nosratinia et al. 2003; Mohr et al.2000; Stockhammer and Buchner 2001; Dumitrescu et al. 2004; Hamzaoui et al.2002; Stankovic et al. 2002] have been proposed to find the optimal UEP/ULP so-lutions. In [Chande and Farvardin 2000], dynamic programming was employed tofind the optimal UEP solution with fixed-length source data blocks. In [Nosratiniaet al. 2003], the authors also considered fixed-length source data blocks and devel-oped an empirical model for optimal source channel rate allocation. In [Mohr et al.2000], Mohr et al. developed a ULP framework with fixed-length channel codingblocks, and used a greedy and iterative search algorithm to find the optimal channelcoding rates, which costs comparatively long execution time. Stockhammer [Stock-hammer and Buchner 2001] gave an O(N2L2) algorithm, where N is the numberof packets and L is the packet length, to find the optimal ULP solutions under theconditions that the operational rate-distortion (R-D) function is convex and thepacket loss probability is monotonically decreasing with respect to the number oflost packets. Dumitrescu [Dumitrescu et al. 2004] presented an algorithm whichunder the same conditions as [Stockhammer and Buchner 2001] reduces the com-plexity to O(NL2). Stankovic [Hamzaoui et al. 2002; Stankovic et al. 2002] furtherreduced the complexity to O(NL) by making use of the rate-based optimal solutionwith a local search for finding the distortion-based optimal solution.

ACM Journal Name, Vol. V, No. N, June 2005.

Layered Unequal Loss Protection with Pre-Interleaving 3

Besides looking for faster algorithms to find the globally optimal ULP solutions,many algorithms [Kim et al. 2003; Mohr et al. 2000; Puri and Ramchandran 1999]on finding the near-optimal ULP solutions or simplifying the ULP architecturehave also been proposed. In [Mohr et al. 2000; Puri and Ramchandran 1999], thegradient search method or Lagrangian optimization approach is used to approxi-mately achieve optimal FEC assignment. Kim et al. [Kim et al. 2003] reduced thecomplexity of the ULP by employing dynamic programming to find the optimalchannel coding rates for each bitplane instead of each channel coding block. Al-though the experimental results in [Kim et al. 2003] show that the scheme can beexecuted much faster than the ULP, its improvement much depends on the numberof bitplanes.

In addition to the high complexity shortcoming, most existing ULP schemes donot consider the minimum image quality requirement, which results in applyingunnecessary ULP process to the early portions of a bitstream whose correspondingreconstructed images are of low quality and thus useless for practical applications.By observing this problem, a hybrid ULP and ELP (HLP) scheme is proposed in[Grangetto et al. 2002]. The basic idea of the HLP is to constrain the early portionsof a progressive bitstream with ELP whose corresponding PSNRs are less than athreshold while ULP is applied to the rest of the bitstream. Although the HLPcan greatly reduce the probability of failure transmission, i.e., below the minimumquality requirement, its complexity is still as high as the ULP.

In this paper, we propose a layered ULP (L-ULP) scheme for fast progressiveimage transmission over packet loss channels. Our proposed L-ULP is able totackle both the minimum quality requirement and the high computation complexityissue. We are able to achieve a complexity of O(nN2) for L-ULP, where n is thenumber of layers and the basic operation is the calculation of one R-D slope andthe corresponding sorting. By choosing different number of layers, the complexityof L-ULP is adjustable. In particular, we use the rate-based optimal solution witha local search to find the average FEC allocation and use the gradient search tofind the FEC solution for each layer. Experimental results show that the executingtime of L-ULP is much faster than the traditional ULP scheme but the averagedistortion is worse. Therefore, we further propose to combine the L-ULP with thepre-interleaving to have an improved L-ULP (IL-ULP) system. By using the pre-interleaving, we are able to delay the occurrence of the first unrecoverable loss inthe source bitstream and thus improve the performance of L-ULP. Experimentalresults show that IL-ULP even outperforms HLP while the complexity of IL-ULPis almost the same as L-ULP.

The contribution of this paper is three folds. First, we propose the L-ULP scheme,which is different from those works in [Stockhammer and Buchner 2001; Dumitrescuet al. 2004; Hamzaoui et al. 2002; Stankovic et al. 2002] in terms of architecture,search method and complexity. In particular, our proposed L-ULP scheme does notstick to the original ULP architecture. Instead, we simplify the ULP architectureby dividing it into several layers. Then, we use the rate-based optimal solution witha local search to find the average FEC allocation and use the gradient search to findthe FEC solution for each layer. Our proposed L-ULP scheme can achieve a low andadjustable complexity of O(nN2) in finding the FEC solution. Second, we take the



characteristics of source coding into account by integrating the L-ULP with the pre-interleaving, which improves the error resilience of the system and thus allows morebits to be allocated to source coding. Third, by extensive experimental results, wefind that the average FEC protection in the optimal unequal loss protection solutionis bounded by the distortion-based optimal ELP solution and the rate-based optimalsolution.The paper is organized as follows. Section 2 introduces the L-ULP scheme.

Section 3 presents the improved L-ULP scheme, i.e., the IL-ULP. Section 4 givesthe numerical results. Finally, conclusion and discussion are given in Section 5.

2. PROPOSED L-ULP SCHEME

2.1 Problem Statement

As stated in Section 1, in the benchmark ULP scheme [Mohr et al. 2000], a majorproblem is that it merely aims at maximizing the expected PSNR and does not haveany control to prevent the occurrence of low PSNR events. It is well-known thata received image with the PSNR value below a certain threshold is useless for anypractical application. For example, the reconstructed standard 512x512 grayscaleLena image with the PSNR value of 22.17 dB shown in Fig. 1 is almost unreadable.Note that such a PSNR threshold can be determined according to human visualsystems (HVS). In this research, for simplicity, we choose 25 dB as the lowestacceptable image quality. This is in line with the rising quality of service (QoS)requirements for multimedia communications. The ULP scheme does not considersuch a minimum PSNR requirement and still allocates different amounts of FECto the early portions of a bitstream whose corresponding PSNRs are less than thethreshold. This results in unnecessary complexity increase and also increasing theprobability of unsuccessful transmission if we consider any image transmission witha PSNR less than the threshold as a failure transmission. Fig. 2 shows a typicalPSNR degradation performance of using ULP to transmit the Lena image coded at0.2 bpp with 47 bytes per packet. It can be seen that a noticeable portion of theULP results which is claimed superior to the ELP (equal loss protection) [Mohret al. 2000] is actually below 25 dB and thus meaningless. Such an observation hasbeen pointed out in [Grangetto et al. 2002].Another major problem of the ULP is its high complexity which has been men-

tioned in Section 1. As shown in the left of Fig. 3, in the ULP different amounts fiof FEC codes are allocated to different streams. Each stream is basically a channelcoding block. A greedy and iterative search algorithm is used in [Mohr et al. 2000]to find the optimal ULP solutions fi, i = 1, 2, . . . , L. Although such a ULP schemeworks fine for small packet size L such as the size of a ATM cell, it is not suitablefor large values of L such as Internet packet sizes since with the increase of L thecomplexity and the execution time of ULP become intolerable.

2.2 L-ULP System Description and Analysis

A straightforward solution is to give different protection for different layers, a bunchof streams, instead of different streams. The architecture of a general layered ULPscheme is shown in the right of Fig. 3. In an LN rectangle where L is the packetlength and N is the number of packets, each row is a channel coding block and each



column is a packet. Let Li and fi denote the number of rows and the allocated FEClength for the i-th layer. Reed-Solomon (RS) codes with 8 bits/symbol are usedas channel codes. An (N, N fi) RS code encodes each segment of N fi sourcesymbols into a channel block of N symbols, and it can correct up to fi symbolloss [Wicker and Bhargava 1994]. A layer is defined as a group of consecutive rowswith the same loss protection choices independent of other rows. The expecteddistortion in mean square error (MSE) at the receiver end can be formulated as

D = d0

ni=1

di Ci, (1)

where d0 is the distortion of not using any packet for reconstruction, n is thenumber of layers, Ci is the probability of correctly decoding the i-th layer, and diis the corresponding distortion gain. We can see that such a layered ULP is evenmore complicate than the conventional ULP [Mohr et al. 2000] because we need tofind not only the optimal FEC allocation fi but also the optimal layer division Li(Assuming n is given.).In [Kim et al. 2003], Kim et al. simplified this problem by mapping each bitplane

into one layer. In this paper, we divide layers according to source coding R-D (rate-distortion) curves. The basic idea is to let each layer have equal distortion gainwhile the first layer must satisfy the minimum quality requirement. The reason wedivide layers with equal distortion gain is just for simplicity, and it also makes senseto practical applications from the QoS point of view. Certainly, we can divide layersin other ways. The source R-D curve Ds(Rs) of a progressive image coding schemesuch as SPIHT can be obtained by extracting some R-D points such as the endpoints of each bitplane during the encoding process followed by linear interpolationbetween neighbor R-D points [Zhang et al. 2003]. Therefore, given the accumulatedsource data amount up to the i-th layer bi, the corresponding distortion di is givenby

di = Ds(bi). (2)

The layer division can be described as

bi =

D1s (dhs ) if i = 1

L(N f) if i = nD1s (d

hs (i 1)ds) if i = 2, . . . , n 1

(3)

where dhs is the highest distortion determined by the minimum quality requirement,

f is the average FEC protection, ds =dhsd

ls

n1and dls = Ds(bn). In particular,

the amount of the first layer source data b1 is obtained according to the minimumquality requirement dhs . Then, we compute the largest amount of source data bnaccording to the average FEC protection f . Based on bn, we can obtain the lowestdistortion bound dls. After this, the n 1 layers are divided based on the idea ofhaving equal distortion gain ds in the range of [d

ls, d

hs ] for each layer. Note that

the actual value of bi may need to change a little during the implementation dueto byte alignment and channel block alignment.In this research we assume the alignment issues can be neglected and we re-



formulate the expected distortion after the layer division as

D(f ) = d0 (d0 d

hs ) C1(f1)ds

ni=2

Ci(fi), (4)

wheref = {f1, f2, . . . , fn}, f1 f2 . . . fn. Ci(fi) can be derived as

Ci(fi) =

fik=0

P (N, k), (5)

where P (N, k) is the probability of having k lost packets out of the total N packets.P (N, k) can be calculated according to the given packet loss model. If we use thepacket loss model with the exponential probability mass function (PMF), the sameas that in [Mohr et al. 2000], P (N, k) can be computed as

P (N, k) =

1pl

e

1

pl

kN

i=Ni=0

1pl

e

1

pl

iN

, (6)

where pl is the mean packet loss rate.From Eqn. (4) we can see that d0, d

hs and ds are either constant or fixed after the

layer division, and the expected distortion can be separated into the n independentunits. A unit i is associated with a variable Ci(fi), which only depends on fiand is independent of the FEC allocation for other layers. Thus, the problem offinding the optimal FEC allocation is the same as optimal bit allocation among nindependent units, which can be solved by processing the falling convex hull of theR-D slopes, i.e., the gradient search method. Details can be found in [Shoham andGersho 1988; Westerink et al. 1988].Note that we have not discussed how to obtain the average FEC protection f in

Eqn. (3). One way is to simply try on all the possible values over the entire range.Fortunately, the work in [Stankovic et al. 2002] points out that the total number ofprotection in the distortion-based optimal solution should not be less than that inthe rate-based optimal solution. Therefore, we can limit the search range for f tobe [f, N 1], where f is the rate-based optimal solution obtained by

f = argmaxf

(N f) C(f). (7)

Based on the analysis above, the proposed L-ULP algorithm, LULP (N, L, n),can be summarized as follows:

Step 1: Find the optimal ELP solution f by using common search methods suchas exhaustive search or bisection search methods. Initialize the search range for

f to be [f, N 1]. Initialize D = d0 andf

= 0.

Step 2: Orderly pick a value for f from the search range according to the rule ofexhaustive search or bisection search.

Step 3: For the selected f , compute bi according to Eqn. (3).

Step 4: Use the gradient search method [Shoham and Gersho 1988; Westerink

et al. 1988] to find the optimal ULP solutionf which minimizes the distortion



shown in Eqn. (4) under the total bandwidth constraint. If D(f ) < D, set

D = D(f ) and

f

=f . Update the search range for f accordingly.

Step 5: If we still have non-empty search range for f , go back to Step 2. Other-

wise, outputf

.

2.3 Complexity Analysis of L-ULP

In L-ULP, the complexity of finding the optimal ELP solution is O(N) because thereare N possible values for f. For each value of f , we need to compute at most nNR-D points and n(N 1) R-D slopes, and sort those R-D slopes to find the equalslope points. Hence, the complexity of the gradient search is O(nN) for one fvalue. Since there are N f possible values for f , the overall complexity for L-ULP is approximately O(nN2), which is much lower than the O(N2L2) algorithmsin [Dumitrescu et al. 2004] and comparable to the O(NL) algorithm in [Stankovicet al. 2002]. Note that the basic operation for the algorithms in [Dumitrescu et al.2004; Stankovic et al. 2002] is the computation of the cost function, which involvesL multiplications and L additions, while the basic operation in our proposed L-ULPis much simpler, i.e., calculating one R-D slope and the corresponding sorting. Inaddition, the complexity of our proposed L-ULP scheme is scalable through settingdifferent numbers of layers n.

3. IMPROVED LAYERED UNEQUAL LOSS PROTECTION

In this section, we propose an improved L-ULP (IL-ULP) scheme to further enhancethe performance of the L-ULP.

3.1 System Description

An interesting channel decoding scheme is reported in [Hagenauer et al. 2000] whichdoes not aim at achieving low error rate, but rather makes the first error as far outas possible under the actual channel condition. Such a channel decoding schemecan be well combined with progressive image coding since the performance of aprogressive image transmission is very much determined by the location of the firstunrecoverable error instead of the amount of errors. This important property hasnot been received enough attention in the area of joint source channel coding forprogressive data transmission. Note that this claim does not strictly hold for theprogressive codec of JPEG-2000 [Taubman and Marcellin 2002]. Since in JPEG-2000 each code-block in a subband is independently coded, with the help of theerror-resilient tools [Moccagatta et al. 2000], the decoder is able to re-start decodingfrom the next code-block if the current one is corrupted. For this case, we arguethat, in a JPEG-2000 bitstream, it still puts the more important data, in terms ofresolution and quality, ahead of the less important data, and thus the location ofthe first unrecoverable error still has significant effects on the overall performance.Motivated by the idea presented in [Hagenauer et al. 2000], in this paper we

propose an improved L-ULP (IL-ULP) scheme, which is a combination of pre-interleaving with our previous proposed L-ULP, for progressive image transmissionover packet loss channels. By using the pre-interleaving, we are able to delay theoccurrence of the first unrecoverable loss in the source bitstream and thus improvethe L-ULP performance. This is the major innovation of our proposed IL-ULP



scheme. Fig. 4 gives an example of the L-ULP with three layers to illustrate thebasic idea of this innovation. The left one in Fig. 4 is the original L-ULP, wherethe source bitstream is placed in the rectangle from left to right and from topto bottom. The right one in Fig. 4 is the L-ULP with pre-interleaving, wherein the rectangle the source bitstream is still placed from top to bottom but ineach layer it is placed in the vertical direction instead of the horizontal direction.Since such an interleaving is performed before channel encoding, we name it aspre-interleaving. The advantage of applying the pre-interleaving can be explainedby the following example. Suppose there exist unrecoverable packet losses and thefirst unrecoverable loss occurs in the j-th packet in the third layer. The positionsof the lost packets can be obtained through checking the sequence number fieldin the header of each packet. For this case, the original L-ULP can use maximal[b1 + b2 + (j 1)] symbols for source decoding while the IL-ULP scheme can takemaximal [b1 + b2 + (j 1) L3] for source decoding, which will result in a betterperformance. The performance gain will be even more significant in the cases ofsmall values of n and large values of L such as Internet packet sizes.Note that the pre-interleaving is not suitable for the traditional ULP scheme

[Mohr et al. 2000]. This is because in the traditional ULP different rows in therectangle have different priorities and in order to provide higher priorities for moreimportant information the source bitstream has to be place from left to right andfrom top to bottom. We would also like to point out the pre-interleaving is notnecessary to be applied to the first layer in the L-ULP, which is designed for theminimum quality requirement, since we consider any source decoding with dataamount less than b1 as a transmission failure. In addition, the pre-interleavingwill only slightly increase the processing complexity and it will not increase anytransmission delay.

3.2 Optimal FEC Allocation

The expected distortion for the proposed L-ULP scheme with the pre-interleavingcan be written as

D(f ) = d0

ni=1

Nfij=1

dji Cji , (8)

wheref = {f1, f2, . . . , fn}, f1 f2 . . . fn, C

ji is the probability of correctly

decoding the j-th packet in the i-th layer and dji is the corresponding distortion

gain. Given fi, dji can be obtained according to source coding R-D curves Ds(Rs).

Cji can be calculated according to the given packet loss model. Compared with the

cost function in Eqn. (1), the cost function in Eqn. (8) is too complicate, whichrequires n(N fi) multiplication and addition. This contradicts our initial objec-tive, i.e., to propose a low complexity L-ULP scheme for fast image transmissionover packet loss channels. Therefore, in this paper, we make a few assumption tosimplify the cost function.In particular, we assume that, for the packet loss model with the exponential

PMF, lost packets are consecutive in an N-packet transmission and it can start atany position with equal probability. This assumption is reasonable since in practicepacket loss caused by network congestion or channel deterioration tends to occur



in bursts. Second, we assume that the R-D relationship within one layer is linear.Based on these two assumptions, we derive the cost function for IL-ULP as

D(f ) = d0

ni=1

di [

fik=0

P (N, k) +1

N fi

Nk=fi+1

P (N, k) N k

2], (9)

where di fi

k=0 P (N, k) is the expected distortion gain due to correctly decoding

the i-th layer and diNfi

Nk=fi+1

P (N, k)Nk2

is the expected distortion gain due to

pre-interleaving in the case of failing to recover lost packets. Since [fi

k=0 P (N, k)+1

Nfi

Nk=fi+1

P (N, k) Nk2

] can be calculated off-line, the complexity of Eqn. (9)

is the same as that of Eqn. (1).Similarly, considering the effect of pre-interleaving, the rate-base optimal solution

f should be

f = argmaxf

(N f)

fk=0

P (N, k) +

Nk=f+1

N k

2 P (N, k). (10)

Then, we directly apply the algorithm LULP (N, L, n), described in Section 2.2,for IL-ULP. The only difference is that for IL-ULP we use Eqn. (9) and Eqn. (10)instead of Eqn. (4) and Eqn. (7).

4. NUMERICAL RESULTS

The six 512x512 images with 8 bits per pixel, Lena, Barbara, Tank, Boat,Goldhill, Peppers, are used as the test images, as shown in Fig. 5. We choosethe packet size of 100 bytes and use the exponential PMF packet loss model. Allprograms were run on a PC with a Windows 2000 operating system, 2.8GHz IntelPentium 4 CPU and 512M RAM. SPIHT is adopted as the codec for source codingand RS codes with 8 bits/symbol are used for channel coding. We choose 25 dB asthe PSNR threshold for the minimum quality requirement. Any image transmissionwith a PSNR value less than the threshold is deemed as a failure transmission.PSNR is typically employed as the image quality measurement in the literature

such as in [Mohr et al. 2000; Kim et al. 2003]. However, in this research, we usethe average distortion for the quality measurement. This is because the maximalaverage PSNR does not correspond to the minimal average distortion due to thelogarithm function. It is well-known that the logarithm function zooms in smallvalues but zooms out large values, which makes the average PSNR not appropriateto represent the expected quality of the received image.Table I and II show the performance comparison of different FEC protection

schemes for transmitting the six test images over the packet-loss channel modelwith a total bandwidth of 0.25 bpp and 0.5 bpp, respectively. The performanceparameters include Average Distortion, Time, Pf and bn. Average Dis-tortion is the expected distortion under the selected FEC solution. Time is theexecuting time to find the optimal FEC solutions. Pf denotes the probability offailure image transmission. bn is the total amount of bytes allocated for sourcecoding under the selected FEC solution.We mainly compare the performance of four schemes: the optimal ELP scheme



(ELP), the HLP scheme [Grangetto et al. 2002], our proposed L-ULP scheme andour proposed IL-ULP scheme. There are three layers in both L-ULP and IL-ULP.Comparing L-ULP with ELP, we can expect that the optimal ELP scheme willhave worse distortion performance but with lower complexity. On the other hand,the HLP considers each row in the left diagram of Fig. 3 as a layer and usesiterative search to find the optimal FEC allocation fi which minimizes the overalldistortion based on Eqn. (1) while it limits the foremost several rows with equalloss protection in order to satisfy the minimum quality requirement. We can expectthat the HLP will have better distortion performance but higher complexity thanthe L-ULP. Therefore, ELP and HLP should be lower and upper bounds in termsof distortion and complexity for our proposed L-ULP scheme.

As shown in Table I and II, the distortion performance of L-ULP is always notworse than that of ELP, up to 37 reduction in distortion. Although the distortionperformance of L-ULP is always not better than that of HLP, the difference issmall, at most 7 in distortion. On the other hand, the executing time of L-ULP,in the order of 101 100, is much faster than that of HLP, in the order of102 103. Moreover, it is surprising that with similar executing time to L-ULP,IL-ULP achieves better distortion performance than HLP at most of the cases, upto 3 reduction in distortion. The reason for this is that pre-interleaving improvesthe loss resilience in the source bitstreams. IL-ULP tends to allocate more bits tothe source coding since it has better loss resilience performance which allows tohave weaker FEC protection. Note that in the case of transmitting Barbara at 0.25bpp, the numerical results look exceptional, i.e., all the four schemes achieve thesame distortion performance. It is because the Barbara image is quite difficult tobe compressed and it needs a lot of data to reach the minimum quality requirement,PSNR 25 dB. Then, at a total bandwidth of 0.25 bpp, there is no much space leftfor unequal protection.

Fig. 6 shows the distortion comparison between ELP, L-ULP, HLP and IL-ULPunder different packet loss rates. It can be observed that the three unequal pro-tection schemes, L-ULP, HLP and IL-ULP always outperform ELP, and the gainbecomes more and more larger with the increase of packet loss rate. The perfor-mance of L-ULP is slightly worse than that of HLP, and the distortion differenceis less than 10. The performance of IL-ULP is the best, even outperforming HLP.

We have also listed the performance of the L-ULP+ scheme and the HLP+LSscheme in Table I and II. The L-ULP+ scheme is almost the same as IL-ULPexcept that in the L-ULP+ we do not consider the pre-interleaving effect in theFEC allocation. In other words, the FEC allocation for L-ULP+ is the sameas that for L-ULP. The HLP+LS scheme is the scheme of applying Stankovicslocal search algorithm (HLP+LS) [Stankovic et al. 2002]. It can be seen thatthe distortion performance of the L-ULP+ scheme is slightly worse than that ofIL-ULP and always better than that of HLP at most of the cases, which furtherdemonstrates the effectiveness of the pre-interleaving. We can also see that theexecuting time of our proposed IL-ULP scheme is slightly shorter than that of theHLP+LS scheme at 0.25 bpp but slightly longer at 0.5 bpp. Overall speaking, thecomplexity of our proposed IL-ULP scheme is comparable to that of the HLP+LSscheme.



By observing the bn results in Table I and II, an interesting finding is that thetotal number of source data in any optimal unequal loss protection solution is notless than that in the optimal ELP solution. Combining this with the finding in[Stankovic et al. 2002], we can further conclude that the average FEC protectionin the optimal unequal loss protection solution should be bounded by [fr , f

d ],where fr and f

d are the rate-based optimal solution and the optimal ELP solution,respectively. This conclusion can be applied to speed up the search for f in Eqn. (3).

5. CONCLUSION AND DISCUSSION

In this paper, we have presented the L-ULP scheme which considers the minimumquality requirement and solves the high complexity problem of ULP. The L-ULPcan be executed in a much faster way than the HLP, usually 100 to 1000 timesfaster, while the distortion performance of L-ULP is worse than that of HLP. Byintegrating L-ULP with the pre-interleaving, we have further proposed the IL-ULP scheme, which achieves better distortion performance than HLP while thecomplexity is as low as that of L-ULP. Therefore, the proposed IL-ULP scheme isvery suitable for practical fast image transmission over packet loss networks.Although we have laid out the framework of the IL-ULP scheme, there are some

issues worth further studying. First, through our experiments, we found that in thecontext of the ELP/ULP architecture the average FEC redundancy in the optimalunequal loss protection solution is bounded by the rate-based optimal solution andthe distortion-based optimal ELP solution. However, theoretical work is needed toprove this finding. Second, it is interesting to apply the pre-interleaving concept toother image and video codecs. Details are elaborated in the following.

5.1 Reflection on Pre-Interleaving

Traditional interleaving is only designed for improving channel coding performanceand does not take the characteristics of source coding into consideration. Whenthere are residual loss/errors remaining after the channel decoding, the residueloss/error patterns may not be preferred by the source decoding. Therefore, if theinterleaving design can consider the characteristics of source coding, i.e., generatethe residue loss/error pattern that matches the characteristics of source coding,the interleaving can also improve the source coding performance. Our proposedIL-ULP scheme has demonstrated this point.Applying the pre-interleaving to the SPIHT codec is quite successful. This is

because SPIHT is a progressive image codec, in which the earlier portions of theprogressive bitstream are always more important than the later parts and the bit-stream is allowed to be truncated in any position. We could foresee the performanceimprovement for other progressive image and video codecs such as JPEG-2000 andMPEG-4 FGS by using pre-interleaving.The pre-interleaving could also be useful for those non-progressive image and

video codecs. For example, for MPEG-4 with re-synchronization words enabled, aslice, i.e., a data segment starting with a re-synchronization word, is the smallestindependent unit in the compressed video bitstream. The MPEG-4 decoding perfor-mance is very much determined by the slice loss ratio. At the same data loss/errorrate, it is easier for the MPEG-4 decoder to handle burst residue loss/errors withinone or several consecutive slices than to handle random residue loss/errors in many



slices. Therefore, MPEG-4 prefers burst residue loss/error patterns than randomloss/error patterns. This is exactly what the pre-interleaving can provide, i.e.,reserving the burst residue loss/errors instead of distributing them in the sourcebitstream.We would like to point out that the pre-interleaving is just a very simple design.

The key idea is to design interleaving with source coding aware. For a particularimage/video codec, there might exist more suitable interleavers.

Acknowledgement

The authors would like to thank the anonymous reviewers for their valuable input.

REFERENCES

Chande, V. and Farvardin, N. 2000. Progressive transmission of images over memoryless noisychannnels. IEEE Journal on Selected Areas in Communications, 850860.

Dumitrescu, S., Wu, X., and Wang, Z. 2004. Globally optimal uneven error-protected packe-tization of scalable code streams. IEEE Trans. on Multimedia, 230239.

Grangetto, M., Magli, E., Marzo, M., and Olmo, G. 2002. Guaranteeing quality of servicefor image transmission by means of hybrid loss protection. In Proceedings of IEEE ICME 2002.Vol. 2. Lausanne, Switzerland, 469472.

Hagenauer, J., Stockhammer, T., Weiss, C., and Donner, A. 2000. Progressive source codingcombined with regressive channel coding on varying channels. In Proceedings of 3rd ITGConference on Source and Channel Coding. 123130. Munich, Germany.

Hamzaoui, R., Stankovic, V., and Xiong, Z. 2002. Rate-based versus distortion-based optimaljoint source-channel coding. In Proceedings of IEEE DCC02. Snowbird, Utah, 6772.

Kim, J., Mersereau, R. M., and Altunbasak, Y. 2003. Error-resilient image and video trans-mission over the Internet using unequal error protection. IEEE Trans. on Image Processing ,121131.

Li, Z. G. and et al. 2003. A unified architecture for real-time video-coding systems. IEEETrans. on Circuits and Systems for Video Technology 13, 472487.

Moccagatta, I., Soudagar, S., Liang, J., and Chen, H. 2000. Error-resilient coding in JPEG-2000 and MPEG-4. IEEE Journal on Selected Areas in Communications, 899914.

Mohr, A. E., Ladner, R. E., and Riskin, E. A. 2000. Approximately optimal assignment for

unequal loss protection. In Proceedings of IEEE ICIP 2000. Vol. 1. 367370.

Mohr, A. E., Riskin, E. A., and Ladner, R. E. 2000. Unequal loss protection: graceful degra-dation of image quality over packet erasure channels through forward error correction. IEEEJSAC speical issue on Error-Resilient Image and Video Transmission 18, 819828.

Nosratinia, A., Lu, J., and Aazhang, B. 2003. Source-channel rate allocation for progressivetransmission of images. IEEE Trans. on Communications, 186196.

Puri, R. and Ramchandran, K. 1999. Multiple description source ocding using forward errorcorrection codes. In Proceedings of Thirty-Third Asilomar Conference on Signals, Systems andComputers. Vol. 1. 342346.

Said, A. and Pearlman, W. A. 1996. A new fast and efficient image codec based on set parti-tioning in hierarchical trees. IEEE Trans. on CSVT 6, 243250.

Schaar, M. and Radha, H. 2001. Unequal packet loss resilience for fine-granular-scalabilityvideo. IEEE Trans. on Multimedia, 381394.

Shoham, Y. and Gersho, A. 1988. Efficient bit allocation for an arbitrary set of quantizers.IEEE Trans. on ASSP 36, 14451453.

Stankovic, V., Charfi, Y., Hamzaoui, R., and Xiong, Z. 2003. Real-time unequal error protec-tion for distortion-optimal progressive image transmission. In Proceedings of IEEE WCNC03.New Orleans, LA, 317321.

Stankovic, V., Hamzaoui, R., and Xiong, Z. 2002. Packet loss protection of embedded datawith fast local search. In Proceedings of IEEE ICIP02. Snowbird, Utah, II165II168.



Stockhammer, T. 2002. Progressive video transmission for packet lossy channels exploiting feed-

back and unequal erasure protection. In Proceedings of IEEE ICIP 2002. 169172. Rochester,NY.

Stockhammer, T. and Buchner, C. 2001. Progressive texture video streaming for lossy packetnetworks. In Proceedings of 11th International Packet Video Workshop.

Stuhlmuller, K., Link, M., and Girod, B. 1999. Scalable internet video streaming with unequalerror protection. In Packet Video Workshop 99.

Taubman, D. S. and Marcellin, M. W. 2002. JPEG2000: Standard for interactive imaging.Proceedings of The IEEE 90, 13361357.

Westerink, P. H., Biemond, J., and Boekee, D. E. 1988. An optimal bit allication algorithmfor subband coding. In Proceedings of ICASSP88. 757760.

Wicker, S. B. and Bhargava, V. K. 1994. Reed-Solomon Codes. IEEE Press.

Wu, Z., Bilgin, A., and Marcellin, W. 2002. Unequal error protection for transmission ofJPEG2000 codestreams over noisy channels. In Proceedings of IEEE ICIP 2002. 213216.Rochester, NY.

Zhang, X. M., Vetro, A., Shi, Y. Q., and Sun, H. 2003. Constant quality constrained rateallocation for FGS-coded video. IEEE Trans. on Circuits and Systems for Video Technology ,121130.

Fig. 1. The reconstructed 512x512 Lena image with PSNR 22.17 dB.



0 20 40 60 80 100 120 137

14

16

18

20

22

24

26

28

30

32

PSNR

(dB)

Number of Lost Packets

Fig. 2. The PSNR performance of using the ULP to transmit the Lena image coded at 0.2 bppwith 47 bytes per packet under different numbers of lost packets. The channel loss model is anexponential PMF (probability mass function) with a mean loss rate of 0.2.

Packet Number1 N

Stre

am

N

um

ber

1234

LL-1 Image Data

FEC

L-2

Packet Number1 N

Stream

N

um

ber

L1

L2

Ln

1234

LL-1Image Data

FEC

Fig. 3. Left: The ULP structure. Right: A general layered ULP structure.



FEC

a network packet

a channel codingblock

N

L1

L2

L3

FEC

a network packet

a channel codingblock

N

L1

L2

L3

Fig. 4. An example of the L-ULP with three layers. Left: the original L-ULP. Right: with thepre-interleaving.



Fig. 5. The six 512x512 test images. From left to right, top to bottom: Lena, Barbara,Tank, Boat, Goldhill, Peppers.



Table I. The performance comparison of different FEC protection schemes for transmitting the sixtest images over the packet-loss channel model with the exponential PMF. The total bandwidthis 0.25 bpp and pl = 0.2.

Image Scheme Average Time Pf bnDistortion (s) (%) (bytes)

Lena ELP 128.96 - 1.96 2300HLP 94.07 763.76 1.16 3277L-ULP 98.70 0.11 1.16 3274IL-ULP 92.80 0.12 1.16 3624L-ULP+ 93.12 0.11 1.16 3274HLP+LS 95.15 0.17 1.05 3071

Barbara ELP 499.38 - 10.57 4700HLP 499.38 1.55 10.57 4700L-ULP 499.38 0.17 10.57 4700IL-ULP 499.38 0.17 10.57 4700L-ULP+ 499.38 0.17 10.57 4700HLP+LS 499.38 0.016 10.57 4700

Tank ELP 137.18 - 4.13 3300

HLP 125.17 458.69 2.12 4023L-ULP 128.79 0.1 1.96 3920IL-ULP 123.27 0.16 2.67 4630L-ULP+ 124.43 0.1 1.96 3920HLP+LS 125.25 0.15 1.96 3959

Boat ELP 218.92 - 2.67 2700HLP 197.46 126.49 2.48 3277L-ULP 200.27 0.08 2.48 3278IL-ULP 196.61 0.05 2.48 3401LULP+ 196.98 0.08 2.48 3278HLP+LS 197.64 0.11 2.29 3182

Goldhill ELP 173.20 - 1.96 2300HLP 137.98 590.3 1.16 3276L-ULP 142.63 0.1 1.16 3244IL-ULP 137.21 0.09 1.16 3304L-ULP+ 137.60 0.1 1.16 3244

HLP+LS 139.85 0.17 1.16 3105

Peppers ELP 166.44 - 1.96 2300HLP 124.61 457.01 1.27 2949L-ULP 129.13 0.09 1.27 2957IL-ULP 122.99 0.08 1.27 3074L-ULP+ 123.69 0.09 1.27 2957HLP+LS 125.39 0.16 1.27 2922



Table II. The performance comparison of different FEC protection schemes for transmitting the sixtest images over the packet-loss channel model with the exponential PMF. The total bandwidthis 0.5 bpp and pl = 0.2.

Image Scheme Average Time Pf bnDistortion (s) (%) (bytes)

Lena ELP 87.00 - 1.14 3300HLP 52.69 5904.1 0.67 6554L-ULP 57.56 2.05 0.67 6562IL-ULP 51.02 1.90 0.59 6485L-ULP+ 51.33 2.05 0.67 6562HLP+LS 54.33 0.63 0.75 6473

Barbara ELP 259.86 - 2.97 5600HLP 236.52 203.6 2.97 6553L-ULP 240.69 0.45 2.97 6552IL-ULP 237.43 0.27 3.08 6668L-ULP+ 236.53 0.45 2.97 6552HLP+LS 239.01 0.3 3.20 6643

Tank ELP 107.44 - 4.12 6500

HLP 90.10 2467.8 1.50 8415L-ULP 95.09 1.68 1.50 8405IL-ULP 87.07 2.47 1.94 9828L-ULP+ 89.06 1.68 1.50 8405HLP+LS 90.54 0.47 1.57 8278

Boat ELP 159.99 - 2.18 4800HLP 117.88 2491.2 1.37 6554L-ULP 122.35 1.17 1.37 6562IL-ULP 116.65 0.82 1.44 7110L-ULP+ 117.50 1.17 1.37 6562HLP+LS 117.90 0.48 1.37 6598

Goldhill ELP 130.13 - 1.14 3300HLP 90.14 4908.7 0.71 6554L-ULP 94.45 2.08 0.71 6563IL-ULP 88.38 1.83 0.71 6976L-ULP+ 89.10 2.08 0.71 6563

HLP+LS 90.46 0.58 0.71 6523

Peppers ELP 106.50 - 1.14 3300HLP 68.65 4826.9 0.75 6553L-ULP 75.52 1.89 0.75 6551IL-ULP 66.92 1.89 0.75 6551L-ULP+ 68.46 1.89 0.75 6551HLP+LS 72.65 0.53 0.75 5898



0.05 0.1 0.15 0.210

20

30

40

50

60

70

80

90Lena

Packet Loss Rate

Dis

torti

on

ELPLULPHLPILULP

0.05 0.1 0.15 0.260

80

100

120

140

160

180

200

220

240

260Barbara

Packet Loss Rate

Dis

torti

on

ELPLULPHLPILULP

0.05 0.1 0.15 0.250

60

70

80

90

100

110Tank

Packet Loss Rate

Dis

torti

on

ELPLULPHLPILULP

0.05 0.1 0.15 0.240

60

80

100

120

140

160Boat

Packet Loss Rate

Dis

torti

onELPLULPHLPILULP

0.05 0.1 0.15 0.240

50

60

70

80

90

100

110

120

130

140Goldhill

Packet Loss Rate

Dis

torti

on

ELPLULPHLPILULP

0.05 0.1 0.15 0.220

30

40

50

60

70

80

90

100

110Pepper

Packet Loss Rate

Dis

torti

on

ELPLULPHLPILULP

Fig. 6. The distortion performance comparison between IL-ULP, HLP, L-ULP and ELP at 0.5bpp. Figures in the order of from left to right and from top to bottom are the results for Lena,

Barbara, Tank, Boat, Goldhill and Peppers, respectively.


Documents

ACM-LULP