Thesis for the degree of Licentiate of Engineering

Source–Channel Coding for Wireless Networks

Niklas Wernersson

Communication TheorySchool of Electrical Engineering

KTH (Royal Institute of Technology)

Stockholm 2006

Wernersson, NiklasSource–Channel Coding for Wireless Networks

Copyright c©2006 Niklas Wernersson except whereotherwise stated. All rights reserved.

TRITA-EE 2006:009ISSN 1653-5146

Communication TheorySchool of Eletrical EngineeringKTH (Royal Institute of Technology)SE-100 44 Stockholm, SwedenTelephone + 46 (0)8-790 7790

Abstract

The aim of source coding is to represent information as accurately as possibleusing as few bits as possible and in order to do so redundancy from the sourceneeds to be removed. The aim of channel coding is in some sensethe contrary,namely to introduce redundancy that can be exploited to protect the informationwhen being transmitted over a nonideal channel. Combining these two techniquesleads to the area of joint source–channel coding which in general makes it possibleto achieve a better performance when designing a communication system than inthe case when source and channel codes are designed separately. In this thesis twoparticular areas in joint source–channel coding are studied: multiple descriptioncoding (MDC) and soft decoding.

Two new MDC schemes are proposed and investigated. The first is based onsorting a frame of samples and transmitting, as side-information/redundancy, anindex that describes the resulting permutation. In case that some of the transmit-ted descriptors are lost during transmission this side information (if received) canbe used to estimate the lost descriptors based on the received ones. The secondscheme uses permutation codes to produce different descriptions of a block ofsource data. These descriptions can be used jointly to estimate the original sourcedata. Finally, also the MDC method multiple description coding using pairwisecorrelating transforms as introduced by Wang et al is studied. A modification ofthe quantization in this method is proposed which yields a performance gain.

A well known result in joint source–channel coding is that the performance ofa communication system can be improved by using soft decoding of the channeloutput at the cost of a higher decoding complexity. An alternative to this is toquantize the soft information and store the pre-calculatedsoft decision values in alookup table. In this thesis we propose new methods for quantizing soft channelinformation, to be used in conjunction with soft-decision source decoding. Theissue on how to best construct finite-bandwidth representations of soft informationis also studied.

Keywords: source coding, channel coding, joint source–channel coding, mul-tiple description coding, soft decoding, permutation codes.

i

Acknowledgements

During my work at KTH, which eventually produced this thesis, I have had thepleasure of receiving help from a number of skilful persons.Most valuable, by far,has been all the help from my guiding star Professor Mikael Skoglund. Mikael hasbeen truly generous with his time which is highly appreciated and as a technicaladviser I consider him second to none.

I would like to thank the rest of the “plan 4” people includingsenior researches,computer support, the administrators and finally my Ph.D. student colleges. Also,a special thank to Tomas “II” Andersson who came up with the idea that lead tomy first publication. I further wish to thank Dr. Norbert Goertz for taking the timeto act as opponent for this thesis.

Finally I would especially like to thank friends and my family (includinggrandmother Ingeborg and the Brattlofs; Greta and the Nordlings) for all theirsupport during the years.

iii

Contents

Abstract i

Acknowledgements iii

Contents v

I Introduction 1

Introduction 11 Source Coding . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.1 Lossless Coding . . . . . . . . . . . . . . . . . . . . . . 21.2 Lossy Coding . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Channel Coding . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 The Source–Channel Separation Theorem . . . . . . . . . . . . . 104 Multiple Description Coding . . . . . . . . . . . . . . . . . . . . 11

4.1 Theoretical Results . . . . . . . . . . . . . . . . . . . . . 124.2 Practical Schemes . . . . . . . . . . . . . . . . . . . . . 13

5 Source Coding for Noisy Channels . . . . . . . . . . . . . . . . . 195.1 Scalar Source Coding for Noisy Channels . . . . . . . . . 205.2 Vector Source Coding for Noisy Channels . . . . . . . . . 215.3 Index Assignment . . . . . . . . . . . . . . . . . . . . . 23

6 Contributions of the Thesis . . . . . . . . . . . . . . . . . . . . . 23References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

II Included papers 33

A Sorting–Based Multiple Description Quantization A11 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A12 Sorting-Based MDQ . . . . . . . . . . . . . . . . . . . . . . . . A13 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A44 Numerical Results . . . . . . . . . . . . . . . . . . . . . . . . . . A6

v

5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A8Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A9References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A11Addendum to the Paper . . . . . . . . . . . . . . . . . . . . . . . . . . A12

B Multiple Description Coding using Rotated Permutation Codes B11 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B12 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . B23 Proposed MDC Scheme . . . . . . . . . . . . . . . . . . . . . . . B2

3.1 CalculatingE[x|ij ] . . . . . . . . . . . . . . . . . . . . . B33.2 CalculatingE[x|i1, · · · , iJ ] . . . . . . . . . . . . . . . . B43.3 The Effect of the Generating Random Matrices . . . . . . B6

4 Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B64.1 Introducing More Design Parameters . . . . . . . . . . . B7

5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B8Appendix A. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B8Appendix B. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B9References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B9

C On Source Decoding Based on Finite-Bandwidth Soft Information C11 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C12 Problem formulation . . . . . . . . . . . . . . . . . . . . . . . . C2

2.1 Re-Quantization of a Soft Source Estimate . . . . . . . . C32.2 Vector Quantization of the Soft Channel Values . . . . . . C42.3 Scalar Quantization of Soft Channel Values . . . . . . . . C5

3 Implementation of the Different Approaches . . . . . . . . . . . .C53.1 Re-Quantization of a Soft Source Estimate . . . . . . . . C63.2 Vector Quantization of the Soft Channel Values . . . . . . C63.3 Scalar Quantization of Soft Channel Values . . . . . . . . C7

4 Simulation results . . . . . . . . . . . . . . . . . . . . . . . . . . C84.1 SOVQ . . . . . . . . . . . . . . . . . . . . . . . . . . . . C84.2 COVQ . . . . . . . . . . . . . . . . . . . . . . . . . . . C9

5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C10References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C10

D Improved Quantization in Multiple Description Coding by Co rrelat-ing Transforms D11 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D12 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . D23 Improving the Quantization . . . . . . . . . . . . . . . . . . . . . D44 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . D65 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D7References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D8

vi

Part I

Introduction

Introduction

In daily life most people in the western world uses applications resulting from whattoday is known as the areas of source coding and channel coding. These applica-tions may for instance be compact discs (CDs), mobile phones, MP3 players, digi-tal versatile discs (DVDs), digital television, voice overIP (VoIP), videostreamingetc. This has further lead to a great interest in the area of joint source–channel cod-ing, which in general makes it possible to improve the performance of source andchannel coding by designing these basic building blocks jointly instead of treatingthem as separate units. Source and channel coding is of interest when for instancedealing with transmission of information, i.e. data. A basic block diagram of thisis illustrated in Figure 1. HereXn = (X1,X2, · · · ,Xn) is a sequence of sourcedata, originating for instance from sampling a continuous signal, and informationabout this sequence is to be transmitted over a channel. In order to do so the in-formation inXn needs to be described such that it can be transmitted over thechannel. We also want the receiver to be able to decode the received informationand produce the estimateXn = (X1, X2, · · · , Xn) of the original data. The taskof the encoderis to produce a representation ofXn and the task of thedecoderis to produce the estimateXn based on what was received from the channel. Insource–channel coding one is in interested in how to design encoders as well asdecoders.

Xn Xn

Encoder DecoderChannel

Figure 1: Basic block diagram of data transmission.

This thesis focuses on the area of joint source–channel coding. An introductionto the topic is provided as well as four produced papers (papers A-D). The organi-zation is as follows: Part I contains an introduction where Section 1 explains the

2 INTRODUCTION

basics of source coding. Section 2 discusses channel codingwhich leads to Section3 where the use of joint source–channel coding is motivated.One particular areain joint source–channel coding is multiple description coding which is introducedin Section 4 and further developed in Papers A, B and D. Section 5 and Paper Cconsider source coding for noisy channels. In Section 6 the main contributions ofPapers A–D will be summarized and finally, Part II of this thesis contains PapersA–D.

1 Source Coding

When dealing with transmission or storage of information this information gener-ally needs to be represented using a discrete value. Source coding deals with howto represent this information as accurately as possible using as few bits as possi-ble, casually speaking “compression.” The topic can be divided into two cases:lossless coding, which requires the source coded version of the source data to besufficient for reproducing an identical version of the original data. When dealingwith lossy codingthis is no longer required and the aim here is rather to reconstructan approximated version of the original data which is as goodas possible.

How to define “good” is not a trivial question. In source coding this is solvedby introducing some structured way of measuring quality. This measure is calleddistortion and can be defined in many ways depending on the context. See forinstance [2] for a number of distortion measures applied to gray scale image cod-ing. However, when dealing with the more theoretical aspects of source codingit is well-established practice to use the mean squared error (MSE) as a distor-tion measure. The dominance of the MSE distortion measure ismore likely toarise from the fact that the MSE in many analytical situations can lead to nice andclosed form expressions rather than its ability to accurately model the absolutetruth about whether an approximation is good or bad. However, in many appli-cations the MSE is a fairly good model for measuring quality and we will in thisentire thesis use MSE as a distortion measure. Assuming the vectorXn containsthen source data values{Xi}ni=1 and the vectorXn contains then reconstructedvalues{Xi}ni=1, the MSE is defined as

DMSE = E

[

1

n

n∑

i=1

(Xi − Xi)2

]

. (1)

1.1 Lossless Coding

AssumeXn, where theXi’s now are discrete values, in Figure 1 describes forexample credit card numbers which are to be transmitted to some receiver. Inthis case it is crucial that the received information is sufficient to extract the exactoriginal data since an approximated value of a credit card number will not be very

1 SOURCECODING 3

useful. This is hence a scenario where it is important that noinformation is lostwhen performing source coding which requires lossless coding.

It seems reasonable that there should exist some kind of lower bound on howmuch the information inXn can be compressed in the encoder. This bound doesindeed exist and can be found by studying theentropy rateof the process that pro-duces the random vectorXn. Let X be a discrete random variable with alphabetAX and probability mass functionp(x) = Pr{X = x}, x ∈ AX . TheentropyH(X) of X is defined as

H(X) = −∑

x∈AX

p(x) log2 p(x). (2)

H(X) is mainly interesting when studying independent identically distributed(i.i.d.) variables. When looking at non–i.i.d. stationary processes theorder-n en-tropy

Hn(Xn) = − 1

n

∑

xn∈AnX

p(xn) log2 p(xn) (3)

and theentropy rateH∞(X) = lim

n→∞Hn(Xn) (4)

are of greater interest. Note that all these definitions measures entropy in bitswhich is not always the case, see e.g. [3]. It turns out that the minimum expectedcodeword length,Ln, per coded symbolXi, when coding blocks of lengthn,satisfies

Hn(Xn) ≤ Ln < Hn(Xn) +1

n(5)

meaning that by increasingn, Ln can get arbitrary close to the entropy rate of arandom stationary process. It can be shown thatHn+1(X

n+1) ≤ Hn(Xn) ∀n andhence, the entropy rate provides a lower bound on the averagelength of a uniquelydecodable code. For the case of non–stationary processes the reader is referredto [4].

There are a number of coding schemes for performing losslesscoding; Huff-man coding, Shannon coding, Arithmetic coding and Ziv-Lempel coding are someof the most well known methods [3].

1.2 Lossy Coding

As previously stated when dealing with lossy coding we no longer have the re-quirement of reconstructing an identical copy of the original dataXn. Considerfor example the situation when we want to measure the height of a person; theexactlength will be a real value meaning that there will be an infinite number ofpossible outcomes of the measurement. We therefore need to restrict the outcomessomehow, we could for instance assume that the person is taller than0.5m andno taller than2.5m. If we also assume that we do not need to measure the length

4 INTRODUCTION

more accurately than in centimeters the measurement can result in 200 possibleoutcomes which will approximate the exact length of the person. Approximationslike this is done in source coding in order to represent (sampled) continuous sig-nals like sound, video etc. These approximations are referred to as quantization.Furthermore, from (2) it seems intuitive that the smaller the number of possibleoutcomes, i.e. the courser the measurement, the fewer bits are required to repre-sent the measured data. Hence, there exists a fundamental tradeoff between thequality of the data (distortion) and the number of bits required per measurement(rate).

Scalar/vector Quantization

Two fundamental tools in lossy coding are scalar and vector quantization. A scalarquantizer is a noninvertible mapping,Q, of the real line,R, onto a finite set ofpoints,C = {ci}i∈I , whereci ∈ R andI is a finite set of indices,

Q : R→ C. (6)

The values inC constitutes the codebook forQ. Assuming|I| gives the cardinalityof I the quantizer divides the real line into|I| regionsVi (some of them mayhowever be empty). These regions are calledquantization cellsand are defined as

Vi = {x ∈ R : Q(x) = ci}. (7)

We think of i as the product of the encoder andci as the product of the decoder

Xi ci

X Encoder Decoder

Figure 2: Illustration of an encoder and a decoder.

as shown in Figure 2. Vector quantization is a straightforward generalization ofscalar quantization to higher dimensions:

Q : Rn → C. (8)

with the modification thatcni ∈ R

n. The quantization cells are defined as

Vi = {xn ∈ Rn : Q(xn) = cn

i }. (9)

Vector quantization is in some sense the “ultimate” way to quantize a signal vector.No other coding technique exists that can do better than vector quantization for agiven number of dimensions and a given rate. Unfortunately the computationalcomplexity of vector quantizers grows exponentially with the dimension making it

1 SOURCECODING 5

infeasible to use unstructured vector quantizers for high dimensions, see e.g. [5, 6]for more details on this topic.

Finally we also mention the term Voronoi region: if MSE is used as a distortionmeasure the scalar/vector quantizer will simply quantize the valuexn to the closestpossiblecn

i . In this case the quantization cellsVi are called Voronoi regions.

Rate/distortion

As previously stated there seems to be a tradeoff between rate, R, and distortion,D, when performing lossy source coding. To study this we definethe encoder as amappingf such that

f : Xn → {1, 2, · · · , 2nR} (10)

and the decodergg : {1, 2, · · · , 2nR} → Xn. (11)

For a pair off andg we get the distortion as

D = E[ 1

nd(Xn, g(f(Xn)))

]

(12)

whered(Xn, Xn) defines the distortion betweenXn andXn (the special case ofMSE was introduced in (1)). A rate distortion pair(R,D) is achievableif thereexistf andg such that

limn→∞

E[ 1

nd(Xn, g(f(Xn)))

]

≤ D. (13)

Furthermore, therate distortion regionis defined by the closure of all achievablerate distortion pairs. Also, therate distortion functionR(D) is given by the in-fimum of all ratesR that achieves the distortionD. For a stationary and ergodic

0 0.5 1 1.50

1

2

3

D

R(D

)

Figure 3: The rate distortion function for zero mean unit variance i.i.d. Gaussiansource data.

6 INTRODUCTION

process it can be proved that [3]

R(D) = limn→∞

1

ninf

f(Xn|Xn):E[ 1n

d(Xn,Xn)]≤D

I(Xn; Xn). (14)

This can be seen as a constrained optimization problem: find thef(Xn|Xn) thatminimizes mutual informationI(Xn; Xn) under the constraint that the distortionis less or equal toD. In Figure 3 the rate distortion function is shown for the wellknown case of zero mean unit variance i.i.d. Gaussian sourcedata. The fundamen-tal tradeoff between rate and distortion is clearly visible.

Permutation Coding

One special kind of lossy coding, which is used in Paper B of this thesis, is per-mutation coding which will be explained in this section. Permutation coding wasintroduced by Slepian [7] and Dunn [8] and further developedby Berger [9] whichalso is a good introduction to the subject. We will here focuson “Variant I” mini-mum mean-squared error permutation codes. There is also “Variant II” codes butthe theory of these is similar to the theory of “Variant I” codes and is therefore notconsidered.

Permutation coding is an elegant way to perform lossy codingwith low com-plexity. Consider the case when we want to code a sequence of real valued random

−3 −2 −1 0 1 2 3

−3 −2 −1 0 1 2 3ξj

ξj

ξ1 ξ2 ξ3

ξ51

n = 3

n = 101

Figure 4: The magnitude of the samples in two random vectors containing zeromean Gaussian source data are shown. In the upper plot the dimen-sion is3 and the lower the dimension is101.

1 SOURCECODING 7

10−3

10−2

10−1

100

0

1

2

3

4

5

6

R

D

Figure 5: Performance of permutation codes (dashed line), entropy coded quan-tization (dash–dotted line) and rate distortion function (solid line) forunit variance i.i.d. Gaussian data. The permutation code has dimen-sionn = 800.

variables{Xi}∞i=1. With permutation coding this sequence can be vector quantizedin a simple fashion such that the blockXn = (X1,X2, · · · ,Xn) is quantized toan indexI ∈ {1, . . . ,M}. To explain the basic idea consider Figure 4 where anexperiment where two random vectorsXn have been generated containing zeromean i.i.d. Gaussian source data. The magnitude of the different samples are plot-ted on thex–axis. The first vector has dimension3 and the second has dimension101. Furthermore, defineξj , j = 1, · · · , n, to be thejth largest component ofXn

and then consider the “mid sample,”ξ2 in the first plot andξ51 in the second. Ifwe imagine that we would repeat the experiment by generatingnew vectors it isclear thatξ51 from this second experiment is likely to be close toξ51 from the firstexperiment. This is also true for the the first plot when studying ξ2 but we canexpect a larger spread for this case. A similar behavior willalso be obtained for allthe otherξj ’s.

Permutation coding uses this fact, namely that knowing the order of the sam-ples inXn can be used to estimate the value of each sample. Therefore, the orderof the samples is described by the encoder. One of the main advantages of themethod is its low complexity,O(n log n) from sorting the samples, which makesit possible to perform vector quantization in high dimensions. In [10] it shown thatpermutation codes are equivalent to entropy coded scalar quantization in the sensethat their rate versus distortion relation are identical whenn→∞. Although thisresult only holds whenn → ∞ the performance tends to be almost identical aslong as intermediate rates are being used for high, but finite, n’s. For high ratesthis is no longer true. Typically there exists some level forthe rate when increasingthe rate no longer improves the performance. This saturation level depends on the

8 INTRODUCTION

size ofn and increasingn moves the saturation level to a higherR, see [9]. Thisis illustrated in Figure 5 where the performances are shown of permutation codes(dashed line), entropy coded quantization (dash–dotted line) as well as the rate dis-tortion function (solid line) for unit variance Gaussian data. For the permutationcoden = 800 was used and the saturation effect starts to becomes visiblearoundR = 3.5 bits/sample. In [11] it is shown that, somewhat contrary to intuition, thereexist permutation codes with finiten’s possessing an even better performance thanwhenn → ∞ and hence also entropy coded scalar quantization. This effect doeshowever depend on the source distribution.

When encoding and decoding in permutation coding there will exist one code-word, for instance corresponding to the first index, of the form

cn1 = (

←n1→µ1, · · · , µ1,

←n2→µ2, · · · , µ2, · · · ,

←nK→µK , · · · , µK) (15)

whereµi satisfiesµ1 ≤ µ2 ≤ · · · ≤ µK and theni’s are positive integers satisfy-ing n1 + n2 + · · ·+ nK = n. All other codewordscn

2 , cn3 , · · · , cn

M are constructedby creating all possible permutations ofcn

1 meaning that there in total will be

M =n!

∏Ki=1 ni!

(16)

different codewords. If the components ofXn are i.i.d. all of these permutationsare equally likely meaning that the entropy of the permutation indexI will equallog2 M . It is a fairly straightforward task to map each of these permutations toa binary number corresponding to a Huffman code. Hence, the rate per codedsymbol,Xi, is given from

R &1

nlog2 M (17)

where “&” means “close to from above”. Also, it turns out that the optimal encod-ing procedure, for a given set{(ni, µi)}Ki=1, is to replace then1 smallest compo-nents ofXn by µ1, the nextn2 smallest components byµ2 and so on. This furthermeans that ordering the components ofXn also will decide the outcome of thevector quantization. This is an appealing property since sorting can be done withO(n log n) complexity which is low enough for implementing permutation vectorquantization in very high dimensions.

Now defineSi = n1 + n2 + · · · + ni andS0 = 0 and assume theni’s to befixed. The optimal choice ofµi, for the MSE case, is then given from

µi =1

ni

Si∑

j=Si−1+1

E[ξj ] (18)

which can be used to design the differentµi’s. Hence, the expected average of then1 smallestξj ’s createsµ1 etc. When designing theni’s we instead assume theµi’s to be fixed. Defining

pi =ni

n(19)

2 CHANNEL CODING 9

gives an approximation of the rate as

R ≈ −K∑

i=1

pi log2 pi. (20)

Using this as a constraint when minimizing the expected distortion results in aconstrained optimization problem giving the optimal choice ofpi as

pi =2−βµ2

i

∑Kj=1 2−βµ2

j

(21)

whereβ is chosen such that (20) is valid. However,pi will be a real value andni

is required to be an integer meaning that we from (21) need to create an approx-imate optimal value ofni. With these equations we can optimize (18) and (21)in an iterative fashion eventually converging in some solution for the parameters{µi, ni}Ki=1. K is found by trying out this iteration procedure for different K ’s(many of them can be ruled out) and the bestK, i.e. theK producing the bestdistortion, is chosen. For a more detailed description of this procedure see [9].

2 Channel Coding

When performing source coding one aims to remove all redundancy in the sourcedata, for channel coding the opposite is done; redundancy isintroduced into thedata in order to protect the data against channel errors. These channel errors can forinstance be packet losses, considered in Papers A, B and D, orbit errors consideredin Paper C. In Figure 6 the nonideality of the channel is modelled as discretememoryless disturbance,p(j|i), when transmitting the indexi and receiving indexj. Note thatj is not necessarily equal toi. Channel coding tries to protect thesystem against these kinds of imperfections.

p(j|i)j

Xi

X Encoder Decoder

Figure 6: Model of transmission over a channel.

There exist theoretical formulas for how much information that, in theory, canbe transmitted over a channel with certain statistics. Thisvalue is called capacityand tells us the maximum number of bits per channel use that can be transmittedover the channel such that an arbitrary low error probability can be achieved. Itshould be noted that the capacity is an supremum which it not necessarily achiev-

10 INTRODUCTION

able itself, this will depend on the channel statistics. Forstationary and memory-less channels the capacity is

C = maxp(x)

I(X;Y ) (22)

which is the well known formula for capacity originating from Shannon’s ground-breaking paper [1].X andY are not required to be discrete in this formula butgenerally when dealing with continuous alphabets a constraint on p(x) in (22) isintroduced such that the power is restricted, i.e.p(x) : E[X2] ≤ P . Furthermore,if the channel has memory Dobrushin [12] derived the capacity for “informationstable channels” (see e.g. [13] for explanation) and Verdu and Han [14] showed ageneral formula valid for any channel.

3 The Source–Channel Separation Theorem

It is now time to combine the results from the discrete sourcecoding theorem, (5),and the channel capacity theorem, (22). The discrete sourcecoding theorem statesthat the dataXn can be compressed to use arbitrarily close toH∞(X) bits percoded source symbol and the channel capacity theorem statesthat arbitrarily closeto C bits per channel use can be reliably transmitted over a givenchannel. Know-ing these separate results the question about how to design the encoder/decoderin a system which needs to do both source and channel coding, as in Figure 6,arises. Since the discrete source coding theorem only depends on the statisticalproperties of the source and the channel coding theorem onlydepends on the sta-tistical properties of the channel one might expect that a separate design of sourceand channel codes is as good as any other method. It turns out that for stationaryand ergodic sources a source–channel code exist whenH∞(X) < C such thatthe error probability during transmission can be made arbitrary small. The con-verse,H∞(X) > C, implies that the error probability is bounded away from zeroand it is not possible to achieve arbitrary small error probability. The case whenH∞(X) = C is left unsolved and will depend on the source statistics as well asthe channel properties.

For nonstationary sources the source–channel separation coding theorem takesan other shape and we need to use concepts like “strictly dominating” and “domi-nation.” This was introduced and explained in [13, 15].

Based on these theoretical results it may appear as if sourceand channel codescould be designed separately. However, this is only true under the assumptionsvalid when deriving the results in (5) and (22). One of these assumptions is theuse of infinitely long codes, i.e.n → ∞. In practice this is not feasible, espe-cially when dealing with real time applications like video streaming or VoIP. Thismotivates the study of joint source–channel coding since for finite n’s it will bepossible to design better source–channel codes jointly than done separately. Thissubject is the main focus in this thesis and in the following sections we will discuss

4 MULTIPLE DESCRIPTIONCODING 11

R1

R2

Xn

Xn1

Xn2

Xn0

f1

f2

g1

g2

g0

Figure 7: The MDC problem illustrated for two channels.

two particular kinds of joint–source channel coding, namely multiple descriptioncoding and source coding for noisy channels.

4 Multiple Description Coding

In multiple description coding (MDC) the total available rate for transmittingsource data is split between a number of different channels.Each of these channelsmay be subject to failure, meaning that some of the transmitted data may be lost.The aim of MDC is then to reconstruct an approximated versionof the source dataeven when only a subset of the used channels are in working state. The problem isillustrated in Figure 7 for two channels. Heref1 andf2 are the encoders used forchannels1 and2 respectively and defined as

fk : Xn → {1, 2, · · · , 2nRk} ∀k ∈ {1, 2}. (23)

Hence, the encoders will useR1 andR2 of the total available rateR = R1 + R2.There will exist three decoders:g1 andg2 used when only the information fromone channel is received andg0 used when the information from both channels arereceived, i.e. both channels are in working state. The decoders are defined as

gk : {1, 2, · · · , 2nRk} → Xn ∀k ∈ {1, 2} (24)

g0 : {1, 2, · · · , 2nR1} × {1, 2, · · · , 2nR2} → Xn. (25)

For the different decoders we define distortions as

Dk = E[ 1

nd(Xn, Xn

k )]

∀k ∈ {0, 1, 2}. (26)

We callD0 thecental distortionandD1 andD2 side distortions.

12 INTRODUCTION

As an example of MDC, consider the case whenX is i.i.d. binary distributedtaking values0 and1 with probability1/2. Also assume the Hamming distance isused as a distortion measure, i.e.d(1, 1) = d(0, 0) = 0 andd(0, 1) = d(1, 0) = 1.Suppose thatD0 = 0 is required,R1 = R2 = 0.5 bits per symbol and the aim isto minimizeD1 = D2 = D. One intuitive approach would be to transmit half ofthe bits on one channel and the other half on the other channel. This would thengiveD = 0.25 (achieved by simply guessing the value of the lost bits). However,in [16] it is shown that one can do better and it is in fact, somewhat surprisingly,possible to achieveD = (

√2− 1)/2 ≈ 0.207.

The MDC literature is vast, theoretical results as well as practical schemes arepresented in the sections below.

4.1 Theoretical Results

One of the first results in MDC was El Gamal and Cover’s region ofachievable quintuples(R1, R2,D0,D1,D2) [17]. This result states that(R1, R2,D0,D1,D2) is achievable if there exist random variablesX0, X1, X2

jointly distributed with sampleX from an i.i.d. source such that

R1 > I(X; X1), (27)

R2 > I(X; X2), (28)

R1 + R2 > I(X; X0, X1, X2) + I(X1; X2), (29)

Dk ≤ E[

d(X, Xk)]

∀k ∈ {0, 1, 2}. (30)

Ozarow [18] showed this bound to be tight for the case of Gaussian sources withvarianceσ2

X (although Ozarow usesσ2X = 1 in his paper) and also derived closed

form expressions for the achievable quintuples which satisfy

D1 ≥ σ2Xe−2R1 (31)

D2 ≥ σ2Xe−2R2 (32)

D0 ≥{

σ2Xe−2(R1+R2) 1

1−(√

Π−√

∆)2if Π ≥ ∆

σ2Xe−2(R1+R2) otherwise

(33)

where

Π = (1−D1/σ2X)(1−D2/σ2

X) (34)

∆ = D1D2/σ4X − e−2(R1+R2). (35)

Studying these equations by settingR1 = R2 we see that there will be a tradeoffbetween the performanceD1,D2 versus the performanceD0. DecreasingD1 andD2 means that we need to increaseD0 and vice versa (can for instance bee seen inFigure 4 of Paper B whereD1 = D2).

4 MULTIPLE DESCRIPTIONCODING 13

Ahlswede [19] showed that the El Gamal–Cover region is tightfor the “noexcess rate for the joint description” meaning the case whenthe best possibleD0

is achieved according toR1 + R2 = R(D0), whereR(D) is the rate distortionformula. In [20] Zhang and Berger constructed a counterexample which showsthat the El Gamal–Cover region is not tight in general. The problem of findinga bound that fully describes the achievable multiple description region for twodescriptors is still unsolved.

In [21] Zamir shows that

D∗(σ2X , R1, R2) ⊆ DX(R1, R2) ⊆ D∗(PX , R1, R2). (36)

Here σ2X is the variance of the source,PX = 22h(X)/2πe whereh(X) is the

differential entropy ofX. D∗(σ2, R1, R2) denotes the set of achievable distortions(D0,D1,D2) when using ratesR1 andR2 on a Gaussian source with varianceσ2.DX(R1, R2) denotes the set of achievable distortions(D0,D1,D2) for the sourceX.

In [22] outer and inner bounds on the achievable quintuples are achieved thatrelate to the El Gamal–Cover region. The multiple description problem has alsobeen extended theK-channel case in [23] as well as in [24, 25] where the area ofdistributed source coding [26, 27] is used as a tool in MDC. Further results can befound in [28, 29].

4.2 Practical Schemes

Also the more practical area of MDC has received considerable attention, seee.g. [30]. Below are a few of the most well known MDC methods explained inbrief.

Multiple Description Scalar Quantizers

In [31] Vaishampayan makes the first constructive attempt atdesigning a practicalMDC scheme, motivated by the extensive information theory research summa-rized in the previous section. The paper considers designing scalar quantizers, formemoryless source data, as encoders(f1, f2) producing indices(i, j). It is impor-tant to note that the quantization intervals off1 andf2 can be disjoint intervals asshown in Figure 8. This will in turn lead to the existence of a virtual encoderf0

created by the indices fromf1 andf2 and anindex assignment matrix, see exam-ples in Figure 9. The index generated fromf1 is mapped to a row of the indexassignment matrix andf2 is mapped to a column. Hence, when both indices arereceived we know that the original source data must have beenin the interval cre-ated by the intersection of the two quantization intervals described byf1 andf2,i.e. x ∈ {x : (f1(x) = i) ∧ (f2(x) = j)}. The virtual encoderf0 will thereforegive rise to the cental distortionD0 which is illustrated in Figure 8 where the leftindex assignment matrix of Figure 9 is used.

14 INTRODUCTION

x

x

x1 2 3 4 5 6f0(x) :

1 2 1 2 2 3f1(x) :

1 1 2 2 3 2f2(x) :

Figure 8: Encodersf1 and f2 will together with the (left) index assignmentmatrix of Figure 9 create a third virtual encoderf0.

.... ..

. ..

. ..

. . .1

2

3

4 5

6 7

8

9

10 11

12 13...

. . .. . .

. . .

. . .1

2

3

4

5

6

7

8

9

10

11 12

13

14

15

16 17

18 21

Figure 9: Two examples of index assignment matrices. The left matrix willenable more protection against packet losses and hence a lower per-formance onD0. The right matrix will on the other hand enable ahigher performance onD0.

Based on this idea the MDC system is created by optimizing thelagrangianfunction

L = E[d(X, X0)] + λ1(E[d(X, X1)]−D1) + λ2(E[d(X, X2)]−D2). (37)

It is shown that this optimization problem results in a procedure where (i) for afixed decoder, optimal encoders can be created, and (ii) for afixed encoder, op-timal decoders can be created. Alternating between these two optimization crite-rions will eventually converge to a solution just as in regular vector quantizationtraining. By choosing low values forλ1 andλ2 the solution will converge to anMDC scheme with high performance onD0 and low performance onD1 andD2.Choosing high values for theλk ’s will on the other hand yield a low performanceon D0 and a high performance onD1 andD2. Hence,λ1 andλ2 can be used todesign the system for different levels of error protection.

Furthermore, also the design of the index assignment matrixwill impact thetradeoff betweenD0,D1 andD2. In order to optimizeD0 there should be noempty cells in the matrix leading to as many quantization regions for the virtualencoderf0 as possible. On the other hand, if we are interested in only optimizingD1 andD2 there should only be nonempty cells only along the diagonal of theindex assignment matrix corresponding to transmitting thesame description on

4 MULTIPLE DESCRIPTIONCODING 15

both channels. In Figure 9 two examples of index assignment matrices are shownand since there are more nonempty cells in the right example using this matrix willmake it possible to get a better performance onD0 than if the left matrix was used.

The difficult problem on how to actually design the optimal index assignmentmatrix is not solved in the paper, instead two heuristic methods to design thesematrices are presented which are argued to have good performances.

This original idea of Vaishampayan has been investigated and improved inmany papers since it was introduced in [31]. In [32] the method is extended toentropy–constrained MDSQ (ECMDSQ). Here, two additional terms are addedin the Lagrangian function (37), similar to what is done in entropy–constrainedquantization (see e.g. [33]), resulting in

L = E[d(X, X0)] + λ1(E[d(X, X1)]−D1) + λ2(E[d(X, X2)]−D2)

+λ3(H(I)−R1) + λ4(H(J)−R2) (38)

whereH(I) andH(J) are the entropy of the indicesi and j generated by theencodersf1 andf2. It is shown that introducing this modification still leads to aniterative way to optimize the system. Hence, alsoλ3 andλ4 can be used to controlthe convergence of the solution, increasing the values of these will try to force thesolution to use a lower rate. A comparison between MDSQ and ECMDSQ can forinstance bee seen in Figure 4 of Paper B whereD1 = D2.

In [34] a high–rate analysis of MDSQ and ECMDSQ is presented.Motivatedby the fact that comparing different MDC schemes is hard due to the many pa-rameters involved (cental/side distortions, rates at the different channels) it is alsoproposed that the productD0D1 for the balanced case, i.e.R1 = R2, is a goodfigure of merit when measuring performance. As a special caseMDSQ/ECMDSQare analyzed for the Gaussian case and then compared to the Ozarow bound (31-33). This resulted in an an 8.69 dB/3.07 dB gap respectively compared to thetheoretical bound. This result was later strengthen in [35].

Some improved results on the index assignment were obtainedin [36] and thisproblem was also studied in [37] where an algorithm is found for designing anindex assignment matrix when more than two channels are used.

Multiple Description Lattice Vector Quantization

The idea of multiple description lattice vector quantization (MDLVQ) is intro-duced in [38, 39]. This is in some sense an extension of the idea of MDSQ tothe vector quantization case. However, when dealing with unconstrained vectorquantization the complexity grows very quickly with the number of dimensions;in order to reduce this complexity the vector quantization can be constrained insome way which generally results in a decreased complexity at the cost of a sub-optimal performance. One example of this is lattice vector quantization where allcodewords are from a lattice (or possibly a subset). This greatly simplifies theoptimal encoding procedure and a lower encoding complexityis achieved. The

16 INTRODUCTION

high complexity of unconstrained vector quantizers implies that a pure multipledescription vector quantization (MDVQ) scheme, suggestedin e.g. [40–42], maybe impractical for high dimensions and rates which motivates the use of latticevector quantization in the framework of MDC.

In ann-dimensional MDLVQ two basic lattices are used: the fine lattice Λ ⊂R

n and the coarser latticeΛ′ ⊂ Rn. Λ will constitute the codewords of the central

decoder andΛ′ will constitute the codewords of the side decoders. Furthermore,Λ′

is chosen such that it is geometrically similar toΛ meaning thatΛ′ can be createdby a rotation and a scaling ofΛ. In addition, no elements ofΛ should lie on theboundaries of the Voronoi regions ofΛ′. An important parameter is the index

K =

∣

∣

∣

∣

Λ

Λ′

∣

∣

∣

∣

(39)

which describes how many lattice points fromΛ there exist in the Voronoi regionsof Λ′ (it is assumed thatK ≥ 1). The lower the value ofK, the more errorprotection is put into the system.

Based on these lattices an index assignment mapping function ℓ, which is aninjection, is created as a one-to-one mapping between a lattice point inΛ and twolattice points inΛ′ × Λ′. Hence,

Λ1−1←→ ℓ(Λ) ⊆ Λ′ × Λ′. (40)

The encoder will start start quantizing a given vectorXn to the closet pointλ ∈ Λ.By derivingℓ(λ) the resulting point is mapped to(λ1, λ2) ∈ Λ′×Λ′, i.e. two pointsin the coarser lattice. It should here be noted that the orderof these points are ofimportance meaning thatℓ−1(λ1, λ2) 6= ℓ−1(λ2, λ1). Descriptions ofλ1 andλ2

are then transmitted over one channel each and if both descriptions are received theinverse mappingℓ−1 is used to recoverλ. If only one descriptor is receivedλ1, orλ2, is used as a reconstruction point. This means that the distance betweenλ andtheλk ’s will affect the side distortion and [39] considers the design of the indexmapping for the symmetric case when producing equal side distortions from equal-rate channels (further investigated in [43] for the asymmetric case). An asymptoticanalysis is also provided which reveals that the performance of MDLVQ can getarbitrarily close to the asymptotic multiple description rate distortion bound [44]when the rate and dimension approach infinity. Also [35] provides insight in theasymptotical behavior of MDLVQ.

In [45] a simple, yet powerful, modification of the encoder isintroduced whichmakes it possible to not only optimize the encoding after thecental distortionwhich was previously the case. This is done by instead of, as in the original idea,minimizing

‖Xn − Xn0 ‖2 (41)

minimize

α‖Xn − Xn0 ‖2 + β(‖Xn − Xn

1 ‖2 + ‖Xn − Xn2 ‖2). (42)

4 MULTIPLE DESCRIPTIONCODING 17

Choosing a large value onβ will decrease the side distortion at the cost of increas-ing the central distortion and vice versa.

Multiple Description Using Pairwise Correlating Transform s

Multiple description coding using pairwise correlating transforms (MDCPC) wasintroduced in [46–49]. The basic idea here is to create correlation between thetransmitted information. This correlation can be exploited in the case of a packetloss on one of the channels since the received packet due to the correlation willcontain information also about the lost packet. In order to do this apiecewisecorrelating transformT is used such that

[

Y1

Y2

]

= T

[

X1

X2

]

, (43)

where

T =

[

r2 cos θ2 −r2 sin θ2

−r1 cos θ1 r1 sin θ1

]

. (44)

Herer1 andr2 will control the length of the basis vectors andθ1 andθ2 will controlthe direction. The transform is invertible so that

[

X1

X2

]

= T−1

[

Y1

Y2

]

. (45)

Based on the choice ofr1, r2, θ1, θ2 a controlled amount of correlation, i.e. redun-dancy, will be introduced inY1 andY2 which are transmitted over the channels.The more redundancy introduced the lower side distortion will be obtained at thecost of an increased central distortion.

However, in their present form (43)-(45) use continuous values. In order tomake the idea implementable quantization needs to be performed at some stagein these equations. This is solved by quantizingX1 and X2 and then findingan approximation of the transformT which ensures that alsoY1 andY2 will bediscrete. In Paper D of this thesis we propose to change the order of this such thatthe transformation is performed first and secondlyY1 andY2 are quantized. Thisresults in a performance gain.

To get some intuition about the behavior of the original method some of thetheoretical results of [48, 50] are reviewed: For the case when R1 = R2 = Rand using high rate approximations it can be showed that whenno redundancy isintroduced between the packets, i.eT equals the identity matrix, the performancewill behave approximately as

D∗0 =πe

6σ1σ22

−2R (46)

D∗s =1

4(σ2

1 + σ22) +

πe

12σ1σ22

−2R (47)

18 INTRODUCTION

whereD∗s is the average side distortion between the two channels. Using thetransformation matrix

T =1√2

[

1 11 −1

]

(48)

we instead get

D0 = ΓD∗0 (49)

Ds =1

Γ2

1

4(σ2

1 + σ22) + Γ

πe

12σ1σ22

−2R (50)

where

Γ =(σ2

1 + σ22)/2

σ1σ2. (51)

Hence, the central distortion is increased (assumingσ1 > σ2) and the constantterm in the average side distortion is decreased at the same time as the exponentialterm is increased. Two conclusions can be drawn; firstly MDCPC is not of interestwhen the rate is very high, sinceDs is bounded by a constant term. Secondly, themethod also requires unequal variances for the sourcesX1 andX2 since otherwiseΓ = 1. The method is however efficient in increasing robustness with a smallamount of redundancy.

The method has been further developed in [50] where it is extended for usingmore than two channels. Some analytical results on the case of gaussian sourcedata are presented in [51].

Multiple Description Coding Using Frames

The idea of multiple description coding using frames has obvious similarities withordinary block channel coding, see e.g. [52]. The idea presented in [53–56] is tomultiply ann-dimensional source vectorXn with a rectangular matrixF ∈ R

m×n

of rankn and withm > n;Y m = FXn. (52)

Y m will constitute an overcomplete representation ofXn andXn is hence de-scribed bym discriptors which can be transmitted over one channel each in theordinary MDC fashion. In the case that at leastn discriptors are receivedXn canbe recovered by creating the (pseudo-)inverse matrix corresponding to the receiveddescriptors inY m. In the case thatq < n descriptors are received the received datawill describe and(n − q)–dimensional subspaceSq such thatXn ∈ Sq. Xn canthen be reconstructed as

xn = E[Xn|xn ∈ Sq] (53)

if the aim is to minimize the MSE [57].In the idea above we have not included the fact that quantization will be nec-

essary of the descriptorsY m. The basic idea is however still valid and the impactof the quantization is analyzed in [58].

5 SOURCECODING FORNOISY CHANNELS 19

Other Methods

An other interesting approach was introduced in [59] where entropy–codeddithered lattice quantizers are used. The method is shown tohave a good perfor-mance but it is not asymptotic optimal. This work is carried on in [60, 61] whichresults in a scheme that can achieve the whole Ozarow region for the Gaussiancase when the dimension becomes large.

In Paper A we study the possibility to use sorting as a tool to produce MDCand somewhat related to this we study the use of permutation codes in an MDCscheme in Paper B.

5 Source Coding for Noisy Channels

In Section 1 we explained the basics of source coding. Here a random variableXn was encoded into an indexi which in turn is decoded to an estimateXn

i ofXn. However, when transmittingi over a nonideal channel imperfections in thechannel may lead to that we receive a valuej which not necessarily equalsi. Thiseffect needs to be taken into consideration when designing asource code that willbe transmitted over a nonideal channel. We introduce the basic concepts by anexample; consider a2–dimensional source distributed as in Figure 10(a) where

X1

X2

(a)

i = 1

i = 2

i = 3

j = 1

j = 2

j = 3

p(1|1) = 1

p(1|2)

= 1

p(3|3) = 1

(b)

Figure 10: (a) Illustration if a2–dimensional distribution and (b) the channelp(j|i).

the source data is uniformly distributed over3 regions. Further assume that weneed to quantize this source to an indexi ∈ {1, 2, 3} which will be transmittedover a channelp(j|i) described by Figure 10(b). In traditional source coding wewould simply aim for minimizing the distortion without invoking channel knowl-edge in the design. This would, in the MSE case, produce quantization regionsand reconstruction points as illustrated in Figure 11(a). Hence, a large contribu-tion to the distortion occurs when transmitting the indexi = 2 since the receiverwill interpret this as ifi = 1 was actually transmitted and reconstruct accordingly.However, invoking information about the channel statistics in the design would

20 INTRODUCTION

X1

X2

j = 1 j = 2 j = 3

i = 1 i = 2 i = 3

(a)

X1

X2

j = 1 j = 3

i = 1 i = 2 i = 3

(b)

Figure 11: Using the MSE distortion measure: (a) Encoder/decoder optimizedfor an ideal channel and (b) encoder/decoder optimized for the non-ideal channel of Figure 10(b).

make it possible to optimize the system better. This will enable the system to bet-ter protect itself against channel failures resulting in the design illustrated in Figure11(b) where a better overall MSE will be obtained than in the previous case. Notethat in the encoding we need to consider both how to design thequantization re-gions as well as the index assignment, i.e. how to label the quantization regionswith indices (changing the labelling affects the performance). The basic problemsof source coding with noisy channels are clear from this example: quantization,index assignment and decoding. These problems are discussed in the followingsections.

5.1 Scalar Source Coding for Noisy Channels

The first constructive work on scalar source coding for noisychannels was carriedout by Fine in [62]. Here a communication system with a discrete noisy channelis considered and rules for creating encoders as well as decoders are presented.This work continues in [63] where optimal quantizers and decoders are developedfor the case of binary channels, both for the unrestricted scalar quantizer as wellas for uniform quantizers. In [64] Farvardin and Vaishampayan extended this re-sult to other channels and they also introduced a procedure to improve the indexassignment. These results are central in this topic and are summarized below.

p(j|i)j

X Xi

f g

Figure 12: Model of a communication system.

The considered system is illustrated in Figure 12 whereX is assumed to bean i.i.d. source andi ∈ {1, · · · ,M}. It is also assumed that MSE is used as a

5 SOURCECODING FORNOISY CHANNELS 21

distortion measure. For a fixed decoderg the encoder is improved in two steps;first the codewords are relabelled such that

E[X|I = i′1] ≤ E[X|I = i′2] ≤ · · · ≤ E[X|I = i′M ]. (54)

This will improve the index assignment although it not necessarily results in theoptimal index assignment. Secondly, using this new index assignment the scalarquantizer inf is optimized such that

f(x) = arg mini

(

E[(x− X)2|I = i])

. (55)

This will result in optimal scalar quantizers for a given index assignment and agiven decoderg since it will consider the fact thatX is a random variable. It is fur-ther shown that (55) can be developed such that the boarders of the optimal scalarquantizer can be found in an analytical fashion. Altogether, these two steps willimprove the performance of the encoder and the algorithm moves on to optimizethe decoder under the assumption that the encoderf is fixed. The optimal encoderis given by

g(j) = E[X|J = j]. (56)

These equations makes it possible to optimize the encoder and decoder in an iter-ative fashion and will result in a locally optimal solution for f andg, which notnecessarily equals the global optimal solution.

5.2 Vector Source Coding for Noisy Channels

Early works that can be categorized as vector source coding for noisy channelsincludes [65, 66] but the first more explicit work can be foundin [67]. Hereoptimality conditions for encoders and decoders are formulated which results inchannel optimized vector quantization(COVQ). Also [68, 69] studies this subjectwhere [69] is good introduction to COVQ. The central resultsof these publicationsare summarized for the MSE case. The system considered is basically the same asin Figure 12 withX replaced byXn andX replaced byXn. For a fixed decoderg the encoder should perform its vector quantization as

f(xn) = arg mini

(

E[(xn − Xn)2|I = i])

(57)

and for a fixed encoderf the decoder should be designed as

g(j) = E[Xn|J = j]. (58)

Also here the design is done by alternating between optimizing the encoder, (57),and the decoder, (58). Note that there is no step for updatingthe index assignmentas in the scalar case but the algorithm usually tends to converge to a good index

22 INTRODUCTION

assignment. However, the initialization of the index assignment usually affects theperformance of the resulting design.

Further work is done in [70] where it is shown that once a COVQ system hasbeen designed the complexity is no greater than ordinary vector quantization. Thisindeed motivates the use of COVQ. However, just as in regularvector quantizationcomplexity will be an issue when the dimension and/or the rate is high and thisis studied in [71] where restricted vector quantizers are designed for noisy chan-nels. These quantizers will in general have a lower performance but also a lowercomplexity.

Soft Decoding

Most of the work done in joint source–channel coding uses a discrete channelmodel,p(jk|ik), arising from an analog channel in conjunction with a hard deci-sion scheme. This is illustrated in Figure 13 whereik is transmitted and distortedby the analog channel such thatrk is received as

rk = ik + wk (59)

wherewk is additive (real–valued) noise. This value is then converted to a discretevalue jk which results in a discrete channel model. However, if it is assumed

p(jk|ik)

jk

wk

rk

Xn Xnikf g

Figure 13: Model of a channel arising from an analog channel in conjunctionwith a hard decision scheme

that the receiver can access and process the analog (or soft)valuesrk the decoderg could be based onrk, instead onjk. Sinceik → rk → jk will constitute aMarkov chain we can conclude thatrk will contain more, or possibly an equalamont of, information aboutik thanjk which means that decoding based onrk

should make it possible to get a better performance than decoding based onjk.For instance in the MSE case the optimal decoder is given as

Xn = E[Xn|Rk = rk]. (60)

Decoding based on soft channel values is often referred to assoft decodingand the idea originates from [72]. Further work is the area includes [73] whereHadamard–based optimal and suboptimal soft decoders are developed. In [74] a

6 CONTRIBUTIONS OF THETHESIS 23

general treatment for soft decoding of noisy channels with finite memory is pro-vided. One of the main results is that the complex algorithmsresulting from thistheory can be written in a recursive manner lowering the complexity. Also in [75]channels with memory is studied.

Using soft decoding results in a high decoding complexity. In [76, 77] thiscomplexity is decreased by quantizing the soft channel values. The quantizationproduces a vectorjqk and is hence a compromise between using the complete softinformationrk and the commonly occurring coarse version,jk, of the informa-tion. The design criteria used is to design uniform scalar quantizers for therm’ssuch that the mutual information betweenrm andjq

m is maximized. We furtherinvestigate how to createjqk by presenting a different approach in Paper C of thisthesis where theory for designing nonuniform quantizers are developed. Thesequantizers are optimal in the MSE sense.

5.3 Index Assignment

When dealing with noisy channels the index assignment of the different codewordsbecomes an important issue to consider. Otherwise, codewords that interchangefrequently may be far apart in the signal space which may cause a large contribu-tion to the distortion. However, the index assignment problem is NP-hard and tofind the optimal solution one would have to run a full search over theM ! possibleorderings of theM codevectors (although some of these permutations can be ruledout). Conducting a full search is not feasible in most practical situations whichhave made the research focus on suboptimal algorithms, withlimited complexity,to solve the problem.

Early publications about the index assignment problem are [78, 79] whereheuristic algorithms are described. In [69, 80] simulated annealing is used to gen-erate good index assignments and in [81] a greedy method called “Pseudo–GrayCoding” is developed which is shown to have a good performance. In [82, 83] theHadamard transform is used as a tool to create, as well as to indicate the perfor-mance of, index assignments for binary symetric channels. The method is demon-strated to have a high performance and a fairly low complexity. This work isextended to cover more general channel models in [84].

6 Contributions of the Thesis

In this thesis we focus on two topics:

• Multiple description coding. In papers A and B we introduce new MDCschemes. These schemes are analyzed as well as simulated. Inpaper Dwe improve the quantization in multiple description codingusing pairwisecorrelation transforms, briefly explained in Section 4.2.

24 INTRODUCTION

• Soft decoding. The main contribution in paper C is the study of how toquantize soft information for later use in a soft decision scheme.

These papers are summarized in the following sections.

Paper A: Sorting–based Multiple Description Quantization

A new method for performing multiple description coding is introduced. Thescheme is based on sorting a frame of samples and transmitting, as side-information/redundancy, an index that describes the resulting permutation. In thecase that some of the transmitted descriptors are lost this side information (if re-ceived) can be used to estimate the lost descriptors based onthe received ones.This can be done since the side information describes the order of the descriptorswithin the frame and hence each received descriptor will narrow down the possiblevalues of the lost ones. The side information is shown to be useful also in the casewhen no descriptors are lost. For the case of a uniform i.i.d.source a closed formexpression for the performance is derived making it possible to analytically opti-mize the choice of system parameters. Simulations conducted shows the scheme tohave a similar performance to multiple description scalar quantization. The mainadvantage of the suggested method is that it has virtually zero design complexity,making it easy to implement and adapt to varying loss probabilities. It also has theadvantage of allowing straightforward implementation of high dimensional MDC.

Paper B: Multiple Description Coding using Rotated Permutation Codes

The problem of designing multiple description source codesfor J channels is ad-dressed. Our proposed technique consist mainly of two steps; first J copies ofthe vectorXn, containing the source data{xi}i=n

i=1 , arerotatedby multiplicationof J random unitary matrices, i.e. one rotation matrix is generated and used foreach channel. Secondly each of the resulting vectors are, independently of theother vectors, vector quantized using permutation source coding. Furthermore, thedecoding of the received information when only one channel is received is doneby applying the inverse rotation to the result of standard permutation decoding;the optimum strategy for combining the information from multiple channels is amore difficult problem. Instead of implementing optimal combining we propose tosimply average the decoded output of the individual channels and then adjust thelength of the resulting vector based on a theoretical analysis valid for permutationcodes. The choice of using permutation codes comes from the fact that their lowcomplexity makes high dimensional vector quantization possible, i.e. largen’s,and our simulations have indicated that the random generation of rotation matricesworks well when the dimension is high. For low dimensions different outcomes ofthe generated rotation matrices seem to yield quite different performance, mean-ing that the random design may not be as appropriate for this case. Hence, anyvector quantization scheme able to perform quantization inhigh dimensions couldpotentially replace the permutation coding in the proposedscheme.

6 CONTRIBUTIONS OF THETHESIS 25

Using i.i.d. zero-mean Gaussian data for thexi’s we evaluated the proposedscheme by comparing it to multiple description scalar quantization (MDSQ) aswell as entropy-constrained MDSQ (ECMDSQ). It was shown that when usingJ = 2 andR = 4 bits/symbol the proposed system outperformed MDSQ. Com-pared to ECMDSQ a performance gain was achieved when optimizing the systemsfor receiving, close to, only one descriptor. Otherwise ECMDSQ had the betterperformance. The main advantages of the proposed method areits relatively lowcomplexity and its ability to easily implement any number ofdescriptions.

Paper C: On Source Decoding based on Finite–Bandwidth Soft Information

Designing a communication system using joint source–channel coding in generalmakes it possible to achieve a better performance than when the source and channelcodes are designed separately, especially under strict delay-constraints. The ma-jority of work done in joint source-channel coding uses a discrete channel model,corresponding to an analog channel in conjunction with a hard decision scheme.The performance of such a system can however be improved by using soft decod-ing at the cost of a higher decoding complexity. An alternative is to quantize thesoft information and store the pre-calculated soft decision values in a lookup table.In this paper we propose new methods for quantizing soft channel information, tobe used in conjunction with soft-decision source decoding.

The issue on how to best construct finite-bandwidth representations of softinformation is further studied and compared for three main approaches: 1) re-quantization of the soft decoding estimate; 2) vector quantization of the soft chan-nel values, and; 3) scalar quantization of soft channel values. We showed analyt-ically that 1) and 2) are essentially equivalent. Also, since 3) is a special case of2) it can only yield similar or worse performance. However, we derived expres-sions that specify the optimal scalar quantizers, and when using designs based onthese a performance close to that of approaches 1) and 2) was achieved. The gainof this suboptimality is a substantially lower complexity which makes the methodinteresting.

Paper D: Improved Quantization in Multiple Description Codi ng by Corre-lating Transforms

Multiple description using pairwise correlation transforms (MDCPC) is studied.Here a pairwise correlating transforms introduces correlation between differentbitstreams. In the case of a lost bitstream, this correlation can be used in order toget an estimate of a lost stream. In this paper we suggest a newapproach for per-forming the quantization in MDCPC. Using the original method the data is quan-tized and then transformed by a matrix operator in order to increase the redundancybetween descriptors. We suggest to reverse the order of these operations: first thedata is transformed and then quantized. We show that this leads to a modificationof the distortion measure. Using the generalized Lloyd algorithm when designing

26 INTRODUCTION

the quantization codebook also leads to a new way to update the codevectors. Themodification makes it possible to improve the shape of the quantization cells and totailor these to the employed transform. Our simulations indicates that the modifiedmethod performs better than the original one when smaller amounts of redundancyare introduced into the transmitted data. For the simulations conducted the modi-fied method gave 2 dB signal–to–distortion gain compared to the original systemwhen no descriptors were lost. The gain decreased to about 0.5-1 dB when theprobability of lost descriptors was increased.

References[1] C. E. Shannon, “A mathematical theory of communication,”The Bell Systems Techni-

cal Journal, vol. 27, no. 3 and 4, pp. 379–423 and 623–656, July and Oct. 1948.

[2] A. Eskicioglu and P. Fisher, “Image quality measures and their performance,”IEEETransactions on Information Theory, vol. 43, no. 12, pp. 2959–2965, dec 1995.

[3] T. M. Cover and J. A. Thomas,Elements of Information Theory. Wiley-Interscience,1991.

[4] T. S. Han and S. Verdu, “Approximation theory of output statistics,”IEEE Transac-tions on Information Theory, vol. 39, no. 3, pp. 752–772, May 1993.

[5] A. Gersho and R. M. Gray,Vector Quantization and Signal Compression. Dordrecht,The Netherlands: Kluwer academic publishers, 1992.

[6] R. M. Gray and D. L. Neuhoff, “Quantization,”IEEE Transactions on InformationTheory, vol. 44, no. 6, pp. 2325–2383, October 1998.

[7] D. Slepian, “Permutation modulation,”Proc. IEEE, vol. 53, pp. 228–236, March 1965.

[8] J. Dunn,Coding for continuous sources and channels, Ph.D. dissertation. ColumbiaUniv., New York, 1965.

[9] T. Berger, F. Jelinek, and J. Wolf, “Permutation codes for sources,”IEEE Transactionson Information Theory, vol. 18, no. 1, pp. 160–169, January 1972.

[10] T. Berger, “Optimum quantizers and permutation codes,”IEEE Transactions on Infor-mation Theory, vol. 18, no. 6, pp. 759–765, November 1972.

[11] V. Goyal, S. Savari, and W. Wang, “On optimal permutation codes,” IEEE Transac-tions on Information Theory, vol. 47, no. 7, pp. 2961–2971, November 2001.

[12] R. L. Dobrushin, “General formulation of Shannon’s main theorem of informationtheory,”Usp. Math. Nauk., 1959, translated in Am. Math. Soc. Trans., 33:323-438.

[13] S. Vembu, S. Verdu, and Y. Steinberg, “The source-channel separation theorem revis-ited,” IEEE Transactions on Information Theory, vol. 41, no. 1, pp. 44–54, January1995.

[14] S. Verdu and T. S. Han, “A general formula for channel capacity,”IEEE Transactionson Information Theory, vol. 40, no. 4, pp. 1147–1157, July 1994.

[15] S. Vembu, S. Verdu, and Y. Steinberg, “When does the source-channel separationtheorem hold?” inProceedings IEEE Int. Symp. Information Theory, June 1994, p.198.

REFERENCES 27

[16] T. Berger and Z. Zhang, “Minimum breakdown degradation in binary source encod-ing,” IEEE Transactions on Information Theory, vol. 29, no. 6, pp. 807–814, Novem-ber 1983.

[17] A. E. Gamal and T. Cover, “Achievable rates for multiple descriptions,” IEEE Trans-actions on Information Theory, vol. 28, no. 6, pp. 851–857, November 1982.

[18] L. Ozarow, “On a source coding problem with two channels and three receivers,”BellSyst. Tech. J., vol. 59, no. 10, pp. 1909–1921, December 1980.

[19] R. Ahlswede, “The rate-distortion region for multiple descriptions without excessrate,” IEEE Transactions on Information Theory, vol. 31, no. 6, pp. 721–726, Novem-ber 1985.

[20] Z. Zhang and T. Berger, “New results in binary multiple descriptions,” IEEE Transac-tions on Information Theory, vol. 33, no. 4, pp. 502–521, July 1987.

[21] R. Zamir, “Gaussian codes and shannon bounds for multiple descriptions,” IEEETransactions on Information Theory, vol. 45, no. 7, pp. 2629–2636, November 1999.

[22] L. A. Lastras-Montano and V. Castelli, “Near sufficiency of random coding for twodescriptions,”IEEE Transactions on Information Theory, vol. 52, no. 2, pp. 681 – 695,February 2006.

[23] R. Venkataramani, G. Kramer, and V. Goyal, “Multiple description coding with manychannels,”IEEE Transactions on Information Theory, vol. 49, no. 9, pp. 2106–2114,September 2003.

[24] S. Pradhan, R. Puri, and K. Ramchandran, “n-channel symmetric multipledescriptions-part i:(n, k)source-channel erasure codes,”IEEE Transactions on Infor-mation Theory, vol. 50, no. 1, pp. 47–61, January 2004.

[25] R. Puri, S. Pradhan, and K. Ramchandran, “n-channel symmetric multipledescriptions-part ii:an achievable rate-distortion region,”IEEE Transactions on In-formation Theory, vol. 51, no. 4, pp. 1377–1392, April 2005.

[26] D. Slepian and J. Wolf, “Noiseless coding of correlated informationsources,”IEEETransactions on Information Theory, vol. 19, no. 4, pp. 471–480, July 1973.

[27] A. Wyner and J. Ziv, “The rate-distortion function for source coding with side infor-mation at the decoder,”IEEE Transactions on Information Theory, vol. 22, no. 1, pp.1–10, January 1976.

[28] R. Ahlswede, “On multiple descriptions and team guessing,”IEEE Transactions onInformation Theory, vol. 32, no. 4, pp. 543–549, July 1986.

[29] F. Fang-Wei and R. W. Yeung, “On the rate-distortion region for multiple descrip-tions,” IEEE Transactions on Information Theory, vol. 48, no. 7, pp. 2012–2021, July2002.

[30] V. K. Goyal, “Multiple description coding: Compression meets the network,” IEEESignal Processing Magazine, vol. 18, pp. 74–93, sep 2001.

[31] V. A. Vaishampayan, “Design of multiple description scalar quantizers,” IEEE Trans-actions on Information Theory, vol. 39, no. 3, pp. 821–834, 1993.

[32] V. A. Vaishampayan and J. Domaszewicz, “Design of entropy-constrained multiple-description scalar quantizers,”IEEE Transactions on Information Theory, vol. 40, pp.245–250, January 1994.

28 INTRODUCTION

[33] P. A. Chou, T. Lookabaugh, and R. M. Gray, “Entropy-constrained vector quantiza-tion,” IEEE Transactions on Signal Processing, vol. 37, no. 1, pp. 31–42, January1989.

[34] V. A. Vaishampayan and J. C. Batllo, “Asymptotic analysis of multiple descriptionquantizers,”IEEE Transactions on Information Theory, vol. 44, no. 1, pp. 278–284,January 1998.

[35] C. Tian and S. S. Hemami, “Universal multiple description scalar quantization: anal-ysis and design,”IEEE Transactions on Information Theory, vol. 50, no. 9, pp. 2089–2102, September 2004.

[36] J. Balogh and J. A. Csirik, “Index assignment for two-channelquantization,”IEEETransactions on Information Theory, vol. 50, no. 11, pp. 2737–2751, November 2004.

[37] T. Y. Berger-Wolf and E. M. Reingold, “Index assignment for multichannel commu-nication under failure,”IEEE Transactions on Information Theory, vol. 48, no. 10, pp.2656–2668, October 2002.

[38] S. D. Servetto, V. A. Vaishampayan, and N. J. A. Sloane, “Multi description latticevector quantization,” inProceedings IEEE Data Compression Conference, Snowbird,UT, March 1999, pp. 13–22.

[39] V. A. Vaishampayan, N. J. A. Sloane, and S. D. Servetto, “Multipledescription vec-tor quantization with lattice codebooks: Design and analysis,”IEEE Transactions onInformation Theory, vol. 47, no. 5, pp. 1718–1734, 2001.

[40] M. Fleming and M. Effros, “Generalized multiple description vector quantization,” inProceedings IEEE Data Compression Conference, March 1999, pp. 3–12.

[41] N. Gortz and P. Leelapornchai, “Optimization of the index assignments for multipledescription vector quantizers,”IEEE Transactions on Communications, vol. 51, no. 3,pp. 336–340, March 2003.

[42] P. Koulgi, S. Regunathan, and K. Rose, “Multiple description quantization by deter-ministic annealing,”IEEE Transactions on Information Theory, vol. 49, no. 8, pp.2067 – 2075, August 2003.

[43] S. N. Diggavi, N. J. A. Sloane, and V. A. Vaishampayan, “Asymmetric multiple de-scription lattice vector quantizers,”IEEE Transactions on Information Theory, vol. 48,no. 1, pp. 174–191, January 2002.

[44] V. A. Vaishampayan, A. R. Calderbank, and J. C. Batllo, “On reducing granular distor-tion in multiple description quantization,” inProceedings IEEE Int. Symp. InformationTheory, August 1998, p. 98.

[45] V. K. Goyal, J. A. Kelner, and J. Kovacevic, “Multiple description vector quantizationwith a coarse lattice,”IEEE Transactions on Information Theory, vol. 48, no. 3, pp.781–788, March 2002.

[46] Y. Wang, M. Orchard, and A. Reibman, “Multiple description image coding for noisychannels by pairing transform coefficients,” inProc. IEEE 1997 1st Workshop Multi-media Signal Processing (MMSP’97), Princeton, NJ, jun 1997, pp. 419–424.

[47] M. T. Orchard, Y. Wang, V. A. Vaishampayan, and A. R. Reibman, “Redundancy rate-distortion analysis of multiple description coding using pairwise correlating trans-forms,” in Proc. IEEE International Conference on Image Processing, 1997, pp. 608– 611.

REFERENCES 29

[48] Y. Wang, M. Orchard, V. Vaishampayan, and A. Reibman, “Multiple description cod-ing using pairwise correlation transforms,”IEEE Transactions on Image Processing,vol. 10, no. 3, March 2001.

[49] Y. Wang, A. Reibman, M. Orchard, and H. Jafarkhani, “An improvement to multi-ple description transform coding,”IEEE Transactions on Signal Processing, vol. 50,no. 11, November 2002.

[50] V. K. Goyal and J. Kovacevic, “Generalized multiple description coding with cor-relating transforms,”IEEE Transactions on Information Theory, vol. 47, no. 6, pp.2199–2224, sep 2001.

[51] S. S. Pradhan and K. Ramchandran, “On the optimality of block orthogonal transformsfor multiple description coding of gaussian vector sources,”IEEE Signal ProcessingLetters, vol. 7, no. 4, pp. 76 – 78, April 2000.

[52] F. J. MacWilliams and N. J. A. Sloane,The Theory of Error-Correcting Codes. Am-sterdam: North-Holland, 1977.

[53] V. K. Goyal, J. Kovacevic, and M. Vetterli, “Multiple description transform coding:Robustness to erasures using tight frame expansion,” inProceedings IEEE Int. Symp.Information Theory, Cambridge, MA, August 1998, p. 408.

[54] ——, “Quantized frame expansions as source-channel codes for erasure channels,” inProceedings IEEE Data Compression Conference, Snowbird, UT, March 1999, pp.326–335.

[55] V. K. Goyal, J. Kovacevic, and J. A. Kelner, “Quantized frame expansions with era-sures,”Applied and Computational Harmonic Analysis, vol. 10, no. 3, pp. 203 – 233,May 2001.

[56] J. Kovacevic, P. L. Dragotti, and V. K. Goyal, “Filter bank frame expansions witherasures,”IEEE Transactions on Information Theory, vol. 48, no. 6, pp. 1439 – 1450,June 2002.

[57] S. M. Kay,Fundamentals of Statistical Signal Processing: Estimation Theory. Pren-tice Hall, 1993.

[58] V. Goyal, M. Vetterli, and N. Thao, “Quantized overcomplete expansions inRn: anal-

ysis, synthesis, and algorithms,”IEEE Transactions on Information Theory, vol. 44,pp. 16–31, January 1998.

[59] Y. Frank-Dayan and R. Zamir, “Dithered lattice-based quantizersfor multiple descrip-tions,” IEEE Transactions on Information Theory, vol. 48, no. 1, pp. 192 – 204, Jan-uary 2002.

[60] J. Chen, C. Tian, T. Berger, and S. S. Hemami, “A new class of universal multipledescription lattice quantizers,” inProceedings IEEE Int. Symp. Information Theory,September 2005, pp. 1803 – 1807.

[61] ——, “Multiple description quantization via gram-schmidt orthogonalization,” Sub-mitted to IEEE Transactions on Information Theory, April 2005.

[62] T. Fine, “Properties of an optimum digital system and applications,”IEEE Transac-tions on Information Theory, vol. IT-10, pp. 443–457, October 1964.

[63] A. Kurtenbach and P. Wintz, “Quantizing for noisy channels,”IEEE Trans. Commun.Techn., vol. COM-17, pp. 291–302, April 1969.

30 INTRODUCTION

[64] N. Farvardin and V. Vaishampayan, “Optimal quantizer design for noisy channels: Anapproach to combined source–channel coding,”IEEE Transactions on InformationTheory, vol. 33, no. 6, pp. 827–838, November 1987.

[65] J. G. Dunham and R. M. Gray, “Joint source and noisy channeltrellis encoding,”IEEETransactions on Information Theory, vol. 27, no. 4, pp. 516–519, July 1981.

[66] J. D. Gibson and T. R. Fischer, “Alphabeth-constrained data compression,” IEEETransactions on Information Theory, vol. 28, no. 3, pp. 443–457, May 1982.

[67] H. Kumazawa, M. Kasahara, and T. Namekawa, “A constructionof vector quantiz-ers for noisy channels,”Electronics and engineering in Japan, vol. 67-B, pp. 39–47,January 1984.

[68] K. A. Zeger and A. Gersho, “Vector quantizer design for memoryless noisy channels,”in IEEE Internationell Conference on Communications, Philadelphia, USA, 1988, pp.1593–1597.

[69] N. Farvardin, “A study of vector quantization for noisy channels,” IEEE Transactionson Information Theory, vol. 36, no. 4, pp. 799–809, July 1990.

[70] N. Farvardin and V. Vaishampayan, “On the performance and complexity of channel-optimized vector quantizers,”IEEE Transactions on Information Theory, vol. 37,no. 1, pp. 155–159, January 1991.

[71] N. Phamdo, N. Farvardin, and T. Moriya, “A unified approach totree-structured andmultistage vector quantization for noisy channels,”IEEE Transactions on InformationTheory, vol. 39, no. 3, pp. 835–850, May 1993.

[72] V. Vaishampayan and N. Farvardin, “Joint design of block source codes and mod-ulation signal sets,”IEEE Transactions on Information Theory, vol. 38, no. 4, pp.1230–1248, July 1992.

[73] M. Skoglund and P. Hedelin, “Hadamard-based soft decoding for vector quantizationover noisy channels,”IEEE Transactions on Information Theory, vol. 45, no. 2, pp.515–532, March 1999.

[74] M. Skoglund, “Soft decoding for vector quantization over noisy channels with mem-ory,” IEEE Transactions on Information Theory, vol. 45, no. 4, pp. 1293–1307, May1999.

[75] N. Phamdo, F. Alajaji, and N. Farvardin, “Quantization of memoryless and Gauss-Markov sources over binary Markov channels,”IEEE Transactions on Communica-tions, vol. 45, no. 6, pp. 668–675, June 1997.

[76] F. Alajaji and N. Phamdo, “Soft-decision COVQ for Rayleigh fadingchannels,”IEEECommunications Letters, vol. 2, no. 6, pp. 162–164, June 1998.

[77] N. Phamdo and F. Alajaji, “Soft-decision demodulation design for COVQ over white,colored and ISI Gaussian channels,”IEEE Transactions on Communications, vol. 48,no. 9, pp. 1499–1506, September 2000.

[78] K. A. Zeger and A. Gersho, “Zero redundancy channel coding in vector quantization,”IEE Electronic Letters, vol. 23, no. 12, pp. 654–655, June 1987.

[79] J. DeMarca and N. Jayant, “An algorithm for assigning binary indices to the codevec-tors of a multi-dimensional quantizer,” inIEEE Internationell Conference on Commu-nications, Seattle, USA, 1987, pp. 1128–1132.

REFERENCES 31

[80] D. J. Goodman and T. J. Moulsley, “Using simmulated annealing to design digitaltransmission codes for analogue sources,”IEE Electronic Letters, vol. 24, no. 10, pp.617–619, May 1988.

[81] K. A. Zeger and A. Gersho, “Pseudo-Gray coding,”IEEE Transactions on Communi-cations, vol. 38, no. 12, pp. 2147–2158, December 1990.

[82] P. Knagenhjelm and E. Agrell, “The Hadamard transform — A tool for index assign-ment,” IEEE Transactions on Information Theory, vol. 42, no. 4, pp. 1139–1151, July1996.

[83] R. Hagen and P. Hedelin, “Robust vector quantization by a linear mapping of a blockcode,”IEEE Transactions on Information Theory, vol. 45, no. 1, pp. 200–218, January1999.

[84] M. Skoglund, “On channel constrained vector quantization and index assignment fordiscrete memoryless channels,”IEEE Transactions on Information Theory, vol. 45,no. 7, pp. 2615–2622, November 1999.