1

Distributed LT Codes

Srinath Puducheri, Jörg Kliewer, and Thomas E. Fuja.

Department of Electrical Engineering,University of Notre Dame, Notre Dame, IN 46556, USA

IEEE International Symposium on Information Theory (ISIT), 2006.

2

Outline

• Introduction

• Description of LT Code

• Decomposition of LT Code

• Simulation Results

• Conclusions

• References

3

Introduction• Consider a scenario:

T: k information symbols

Distributed LT

Distributed LT

LTN

S1: k/2 information symbols

S2: k/2 information symbols

4

Description of LT Code

• Luby Transform (LT) Codes:– Encoding process

• For the ith encoded symbol, select degree di by Soliton distribution

• Choose di source symbols

• Perform XOR on chosen symbols

– Decoding process• Decode degree-one encoded symbols

• Remove degree-one edges iteratively…

x1 x2 x3 x4 x5 x6

y1 y2 y3 y4 y5

x1

x3

x2

x2

x5

x3

x5

x6

Degree 1 2 3 … k

probability μ1 μ2 μ3 … μk

x4

5

Description of LT Code

• LT process:

A B D

y1 y2 y4

A

B

A B

D

covered: in the ripple S

processed unprocessed: LC

The most likely degree of encoded symbols that get released: k/Ly3

A

C

D

6

Description of LT Code

• The intuition for the degree distribution:– More small degree than large degree.

• Easy to be decoded.

– One single degree-one encoded symbol can trigger the LT process.

• Ideal Soliton Distribution:

k: number of total original symbols

i: degree of encoded symbols

7

Description of LT Code• Ideal Soliton Distribution:

probability

degree

8

Description of LT Code

• In practice:

– Only one single degree-one encoded symbol is too fragile.

• Number of degree-one encoded symbol has to be more.

• A larger Ripple.

– Large degree encoded symbols are necessary evils.

• Because of a larger Ripple.

• To give a better coverage of original symbols.

– A modified Soliton Distribution, concerning the size of Ripple during the LT process.

• Robust Solition Distribution.

9

Description of LT Code

• LT process:

C FD

y5

C

D

E

F

covered: in the ripple S

processed unprocessed: LE…

When L = S, we expect all the original symbols to be covered.

We need the encoding symbols that get released then must cover S original symbols by degree: k/S

10

Description of LT Code

• Robust Soliton Distribution (RSD):

– Robust Soliton Distribution μ(·) is given by:

k: number of total original symbols

i: degree of encoded symbols

11

Description of LT Code• ρ(·) is a distribution over 1 ≦ i ≦ k, and for constants c >

0, 0 <δ< 1, the τ(·) is:

, where S define the size of Ripple:

12

Description of LT Code• Robust Soliton Distribution

K: number of total original symbolsd: degree of encoded symbols

τ( k/S )

probability

k = 10000

13

Decomposition of LT Code• Recall:

S1: k/2 information symbols

S2: k/2 information symbols

T: k information symbols

Distributed LT

Distributed LT

LT

Deconvolved Soliton Distribution

Deconvolved Soliton Distribution

Robust Soliton Distribution

N

14

Decomposition of LT Code• Proposed scenario:

– Each data set contains k/2 information symbols.

– Node N only performs one of two operations:

• Transmits the bitwise XOR of the symbols receives from S1 and S2.

• Transmits just one of them.

• The resulting encoded symbol received by T:

– The degree of the resulting symbol has two cases:

• The sum of the degrees (up to i = k).

• The individual degree of one of the received symbols.

15

Decomposition of LT Code• In determining p(·), we employ the “deconvolution” of the

RSD μ(·).

– If node N XORs the incoming symbols, the resulting degree is th

e sum of the in coming symbols.

– If the degrees are independently chosen according to p(·), their s

um will follow the distribution p * p.

• ‘*’ denotes the discrete-time convolution operation:

16

Decomposition of LT Code• However, a direct attempt at deconvolving μ(·) does not

yield satisfactory results.

– μ(0) = 0, it is necessary that p(0) = 0.

• We cannot permit a degree of zero.

– It implies that the smallest degree in p * p is two.• However, this does not match the RSD as μ(1) > 0.

• degree = 1 would have problem.

( )

17

Decomposition of LT Code• Even if we neglect μ(1), and attempt to deconvolve the re

maining part of μ(·) by solving for p(·) from:

– owing to the fact that μ(i) has a spike at i = k/S, we obtain negati

ve values of p(·) for some arguments.

• degree = k/S would have problem.

18

Decomposition of LT Code• An alternative approach (1/3):

: ρ: τ

Prob.

Deg.

Deg.Deg = 1 Deg = k/S

Deg = k/S

Deg = 1

μ(i)

: ρ: τ

Prob.

Deg.

μ’(i)

: ρ: τ

Prob.

μ’’(i)

19

Decomposition of LT Code• An alternative approach (2/3):

– Split RSD μ(·) into two distributions – μ’(·) and μ’’(·).

with the normalization factor β’ given by:

Deconvolving μ’(·) yields a valid probability distribution function:

20

Decomposition of LT Code• An alternative approach (3/3):

– Split RSD μ(·) into two distributions – μ’(·) and μ’’(·).

with the normalization factor β’’ given by:

and:

21

Decomposition of LT Code• ∵ β’ + β’’ = β, the RSD can be rewritten as:

Hence, the RSD μ(·) can be viewed as a mixture of the

distributions μ’(·) and μ’’(·) with mixing parameter β’/β’’.

22

Decomposition of LT Code• Then we deconvolve μ’(·) to yield a valid probability distri

bution f (·).– By solving:

– Using the same approach as earlier mentioned (recall), with p(·) replaced by f (·) and μ(·) by μ’(·), we obtain the following set of recursive equations:

23

Decomposition of LT Code• The Deconvolved Soliton Distribution (DSD) p(·) is given

by:

with the parameter, given by:

p(·) can be viewed as the mixture of f (·) and μ’’(·) with

mixing parameter λ.

24

Decomposition of LT Code• Notice that:

μ':1 2 … k/S k/2 k

f :1 2 … k/S k/2 k

1 2 … k/S k/2f * f :

k/2 + 1tail

Deg = k/S

: ρ: τ

Deg.

μ(i)Prob.

25

Decomposition of LT Code• Encoding procedure:

D1: k/2 information symbols

DSD : p(·)

encoded symbols: Y

Robust Soliton Distribution

N

D2: k/2 information symbols

S1

S2DSD p(·)

encoded symbols: X1

encoded symbols: X2

Node N knows the degrees and neighbors of X1 and X2

Y = X1 X2

orY = X1

orY = X2

T

DLT-2 Code

26

Decomposition of LT Code• Node N generates a symbol Y as follows:

otherwise (1/2):Generates independent random variables: U1 and U2

uniformly on the interval (0, 1). Based on the values of U1

and U2 , one of the following operations is performed.

27

Decomposition of LT Code• otherwise (2/2):

28

Decomposition of LT Code• The code obtained by combining the two DLT-2 Codes generat

ed by S1 and S2 is not, strictly speaking, an LT Code.

29

Simulation Results• An application of LT Code decomposition:

T: k information symbols

Distributed LT Distributed LT

LTN

S1: m information symbols

SJ: m information symbols

… … …J ≧ 2 source nodes -

k = Jm

Lossy channel

30

Simulation Results• Two schemes (1/2):

– Let every source employ an LT code for its data, and these LT

codes are multiplexed in time at N and sent to T.

• Works well except when source data blocks are small (500-1000

information symbols).

• Overhead is fairly large (about 15-20%).

31

Simulation Results• Two schemes (2/2):

– It would be beneficial if we could form an LT code over the entire

set of k = Jm information symbols at N and send this to T.

• J-fold deconvolution of the RSD.

– Due to the constraint that N has limited processing and storage capabili

ties.

• Code symbols from different sources are XORed together at N.

• It yields symbols that obey the RSD over the range of degrees 1 thr

ough k = Jm.

32

Simulation Results• Consider the case J = 4 (1/2):

– The DLT-2 code can be decomposed further into two more subc

odes called DLT-4 codes.

– By performing a deconvolution of the DSD p(·) similar to how the DSD was derived from the RSD:

• Write DSD p(·) as a mixture of two distributions p’(·) and p’’(·), whe

re p’’(·) contributes to the degrees 1 and k/S.

• The distribution p’(·) is then deconvolved to get g(·) and the final de

gree distribution q(·) is obtained by taking the appropriate mixture of

g(·) and p’’(·).

33

Simulation Results• Consider the case J = 4 (2/2):

– By performing a deconvolution of the DSD p(·) similar to how the DSD was derived from the RSD:

• The degree distribution q(·) is used by source nodes to generate encode symbols from their data, and these code symbols constitute th

e DLT-4 codeword.

• The node N performs a selective XOR of the four DLT-4 code symbols received from four sources, and forwards the resulting code sy

mbol to T.

• Node T needs to collect slightly greater than k = Jm encode symbol

s from N to recover all source data.

34

Simulation Results• Compare two schemes (1/2):

– In terms of the number of code symbols that must be received b

y the sink T to recover all the sources’ data, for J = 4, m = 500, k = Jm = 2000:

• Scheme A:– Each source generates LT code symbols from its 500-symbol d

ata set.

– The code symbols from different sources are time-multiplexed at N and transmitted to the sink T.

– The sink separately decodes each LT code to recover the data.

– Let NA denote the total number of code symbols that are neede

d by T to recover all the data.

35

Simulation Results• Compare two schemes (2/2):

• Scheme B:– Each source generates DLT-4 codewords from its 500-symbol

data set.

– At every instant, the four encode symbols generated by the sources are selectively XOR-ed at N and forwarded to the sink T.

– The sink thus receives a big LT code over all the sources’ data, which is then decoded to recover all the data simultaneously.

– Let NB denote the total number of code symbols that are neede

d by T to recover all the data.

36

Simulation Results• The average values of NA and NB obtained via simulations:

P.S. Probable Max Degree of RSD in Scheme A: up to 500 Probable Max Degree of RSD in Scheme B: up to 2000

37

Simulation Results

38

Conclusions• Presents a novel technique that allows distributed users t

o encode their data by selectively XORed, and results in

an efficient rateless code, similar to an LT Code, at a common destination.

• Exploits the improved efficiency for large blocklengths: by forming a single codeword at the destination representi

ng all the distributed data sets.

– A reduction in overhead compared with a system in which each data set is encoded with its own LT code having a smaller blocklength.

39

References• M. Luby, “LT Codes,” Proc. of the 43rd Annual IEEE Symp.

on Foundations of Comp. Sc., pp 271-280, Vancouver, Canada, November 2002.

• D.J.C. MacKay, “Fountain Codes”, IEE Proceedings – Commun. Vol. 152, No 6, December 2005.

• Ashish Khisti, “Tornado Codes and Luby Transform Codes,” Course Lecture of “6.454: Graduate Seminar in Area I” at MIT, Fall 2003.