Rateless Coding with Feedback

1

Rateless Coding with Feedback

Andrew Hagedorn, Sachin Agarwal , David Starobinski, and Ari Trachtenberg

Department of ECE, Boston University, MA, USA

IEEE INFOCOM 2009

2

Outline

1. Introduction2. Problem Definition3. Shifted LT (SLT) Codes4. Experimental Results5. Conclusion

3

Partial Information

• Transmission Channel with Erasures

Transmitter Receiver

Input symbols Received Symbols

4

Partial Information




5

Partial Information




6

Partial Information




7

Partial Information




8

Partial Information




9

Partial Information

• Multiple Receivers may have different erasures

Transmitter

Receiver 1

Receiver 2

Receiver 3

Given the situation of multiple receivers having partial information, how can all of them be updated to full information efficiently, and over a broadcast channel?

10

Partial Information• Multiple mobile devices may have out-

dated informationa. Mobile databasesb. Sensor network information aggregationc. RSS updates for devices

Broadcaster

Mobile device 1

Mobile device 2

Mobile device 3

Latest version of information

11

Problem Definition•Given an encoding host with k input symbols and a decoding host with n out of the k input symbols, the goal is to efficiently determine the remaining k-n input symbols at the decoding host.

– The encoding host has no information of which k-n input symbols are missing at the decoding host

– Different decoding hosts may be missing different input symbols

•Efficiency1.Communication complexity – Information transmitted from the encoding host to the decoding host should be close in size to the transmission size of the missing k-n input symbols

2.Computational complexity – The algorithm must be computationally tractable

12

• Contribution of this paper – Show that a small amount of feedback, whereby

receivers periodically inform the broadcasting sources about the number of successfully decoded input packets, can lead to major communication, memory, and energy usage gains through a judicious modification of the encoding procedure.

13

Rateless Codes - EncodingUsed for content distribution over error-prone channels

Random choice of edges based on a probability density function

At least k Encoded Symbolsk input symbols

1 =A+B

2 =B

3 =A+B+C

4 =A+C

A

B

C

14

Rateless Codes - DecodingUsed for content distribution over error-prone channels

At least k Encoded Symbols

1 =A+B

2 =B

3 =A+B+C

4 =A+C

k input symbols

SolveGaussian Elimination, Belief Propagation

System of Linear Equations

Irrespective of which encoded symbols are lost in the communication channel, as long as sufficient encoded symbols are received, the decoding can retrieve all the k input symbols

A

B

C

15

Decoding Using Belief Propagation

Decoded k Input Symbols

k+ Encoded Symbols

Decoding host

Redundant!

Decode

Input Symbols

16

Digital Fountain CodesLT Codes

1. Class of rateless erasure codes invented by Michael Luby1

2. Computationally practical (as compared to Random Linear Codes)

3. Fast decoding algorithm based on Belief propagation instead of Gaussian Elimination

4. Form the outer code for Raptor Codes3, which have linear decoding computational complexity

5. Designed for the case when no input symbols are available at the Decoding host initially

•Asymptotic Properties2

•Expected number of encoded symbols required for successful decoding

•Expected decoding computational complexity

•k: number of input symbols

2Assuming a constant probability of failure

)ln( 2 kkOk

1Michael Luby, “LT codes,” in The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002, pp. 271–282.3Amin Shokrollahi, “Raptor codes,” IEEE Transactions on Information Theory, vol. 52, no. 6, 2006, pp. 2551–2567.

)ln( kkO

17

Digital Fountain CodesLT Codes’ Robust Soliton Probability Distribution

• Robust Soliton Probability Distribution k, • Probability of an encoded symbol with degree d is k(d) • Property of releasing degree 1 symbols at a controlled, near-constant rate throughout the decoding process

0 200 400 600 800 1000-6

-5

-4

-3

-2

-1

0

Degree

log 10

(Pro

babi

lity)

LT Code (Robust Soliton)

LT code distribution, k = 1000, c = 0.01, = 0.5.

18

Real-Time Oblivious Erasure Correcting

Amos Beimel, Shlomi Dolev, and Noam Singer

IEEE-Information Theory Workshop 2004, San Antonio, Texas[3] Amos Beimel, Shlomi Dolev, and Noam Singer, “Rt oblivious erasure

correcting”, IEEE/ACM Trans. Netw., vol. 15, no. 6, pp. 1321–1332, 2007.

19

Traditional Erasure Codes

k message

Decodingk message

Encoding

n>k symbols

Transmission Channel

Sender

Receiver

k received X X X X X X

Rate-Less Codesn can be ∞

20

Motivation• Problem

– Channels with high loss rate– Expensive feed-back channels– Weak receiving devices

• Current solutions– ARQ – Requires large feed-back– Erasure Codes – Higher Encoding/Decoding complexity,

a single feedback message• Our goal

– Combine their benefits.

21

Real-Time Codes• Complexity

– Fast symbols generation– Efficient message decoding– Balanced decoding over the entire transmission

• Decoding rate– Rate in which

symbols are decoded

22

Protocol Description

Encoded Symbols

Feed-back

d=3

Calculate degree dRandomly pick d symbolsXOR these symbolsTransmit encoded symbols

Check if exactly 1 symbol missingIf so, decode the missing symbolDump the encoded symbolTransmit the number of decoded symbols r

r=4

rk

kd 1

d=4

23

Conclusions of RT Codes

• A combined approach between ARQ and Erasure Codes

• Low memory overhead• Low feedback - O(√k) messages

24

Inefficiency of LT Codes for our Problem

k+ Encoded Symbols

Decoding host

Decode

Input Symbols

n out of k input symbols are known a priori at the decoding host

Many redundant encoded symbols

Inefficiency of LT Codes for our Problem

•The number of these redundant encoded symbols grows with the ratio of input symbols known at the decoder (n) to the total input symbols (k)

•If n input symbols are known a priori, then an additional LT-encoded symbol will provide no new information to the decoding host with probability

…which quickly approaches 1 as n → k

d

i

k

dk ik

ind01

)(

25

26

Intuitive Fix • n known input symbols serve the function of degree 1

encoded symbols, disproportionately skewing the degree distribution for LT encoding

• We thus propose to shift the Robust Soliton distribution to the right in order to compensate for the additional functionally degree 1 symbols

• Questions – 1) How?– 2) By how much?

0 200 400 600 800 1000-6

-5

-4

-3

-2

-1

0

Degree

log 10

(Pro

babi

lity)

LT Code (Robust Soliton)

27

Shifted Code Construction• Definition

The shifted robust soliton distribution is given by

– k : the number of input symbols in the system– n : the number of input symbols already known at the decoder– round(・ ) rounds to the nearest integer

• Intuitionn known input symbols at the decoding host reduce the degree of each encoding symbols by an expected fraction

j

kn

iij nknk

1roundfor )()(,

kn1

1

28

Shifted Code Distribution

0 200 400 600 800 1000-6

-5

-4

-3

-2

-1

0

Degree

log 10

(Pro

babi

lity)

LT Code (Robust Soliton)Shifted Code

LT code distribution and proposed Shifted code distribution, with parameters k = 1000, c = 0.01, = 0.5. The number of known input symbols at the decoding host is set to n = 900 for the Shifted code distribution. The probabilities of the occurrence of encoded symbols of some degrees is 0 with the shifted code distribution.

29

Shifted Code – Communication Complexity

nknkOnkm 2ln)(

Lemma IV.2 A decoder that knows n of k input symbols needs

encoding symbols under the shifted distribution to decode all k input symbols with probability at least 1−.

ProofWe have k-n input symbols comprising the encoded symbols after the n

known input symbols are removed from the decoding graph. The expresson follows from Luby‘s analysis.

30

Shifted Code – Average Degree of Encoded Symbol

• Lemma IV.3 – The average degree of an encoding node under the k,n distribution is

given by

• Proof– The proof follows from the definitions, since a node with degree d in

the μk distribution will correspond to a node with degree roughly

in the shifted code distribution.From Luby‘s analysis,the expresson for the average degree of an LT

encoded symbol is

)ln( nk

nkkO

kn

d

1

)(ln kO

31

Shifted Codes – Computational Complexity•Lemma IV.4

–For a fixed , the expected number of edges E removed from the decoding graph upon knowledge of n input symbols at the decoding host is given by

E = O (n ln(k − n))•Theorem IV.5

–For a fixed probability of decoding failure , the number of operations needed to decode using a shifted LT code (SLT) is

O (k ln(k − n)) *Proof described in: S. Agarwal, A. Hagedorn and A. Trachtenberg, “Rateless Codes Under Partial Information”, Information Theory and Applications Workshop, UCSD, San Diego, 2008

32

Heuristics for practical implementation

• 1) Non-uniform restriction on feedback– In fact, most input symbols are decoded after n

surpasses a certain value n = αk, 0 ≤ α ≤ 1. – A feedback message containing the most recent

value of n is sent only when the average degree changes by a constant (since the previous feedback).

– When n < nNU, the average degree of an encoding symbol increases by

33


– We limit the feedback to every time the average degree changes by √ log k (from its value at the previous feedback), leading to approximately 1/2√ k feedbacks (obtained by dividing (4) by √logk).

– When n ≥ nNU , the heuristic sends at most √ k feedbacks, one each time the degree changes by (at least) √ log k.

– This heuristic sends O( √ k) feedbacks as n increases from 0 to k, which is equal to the RT code’s feedback.

34


• 2) Uniform restriction on feedback– The current value of n is communicated back to

the encoder every time n increases by √k, resulting in √k feedbacks as n increases from 0 to k

– This heuristic has the advantage of not congesting the feedback channel toward the end of decoding, unlike RT codes and the non-uniform restriction on feedback.

35

Fig. 1. Feedback strategies for uniform and non-uniform restrictions on Shifted LT and RT codes. Each circle qualitatively corresponds to a situation in which the current value of n is fed back to the encoder.

36

Simulation Results

• c = 0.9 and δ = 0.1• In each round of the simulation an encoded

packet is generated and transmitted, and decoding is attempted on the received packet (as well as any stored in memory) at the decoder.

• If an input symbol is recovered then feedback is sent as dictated by each code.

37

Simulation Results

• For k=500, on average Shifted LT codes requires 59% less redundancy than RT codes and 21% less redundancy than LT codes (on average, over 100 trials).

• The feedback channel communication complexity for Shifted LT codes is greater than either RT codes or LT codes.

• While RT codes is limited by the changes in its degree and LT codes transmits no feedback, the Shifted LT code transmits feedback every time it recovers one or more input symbols.

38

Memory usage

Fig. 2. Memory usage at the decoder as a function of the number of transmitted symbols.

39

Number of encoded symbols required

Fig. 3. Number of encoded symbols required to disseminate all k input symbols.

40

Number of feedback messages sent

Fig. 4. The number of feedback messages sent for the different codes for increasing number of input symbols k. The “Shifted LT - no restriction” transmits too many (O(k)) feedbacks and has been left out of this figure.

41

Number of encoded symbols needed

Fig. 5. The number of encoded symbols needed to decode 100 input symbols, as a function of the feedback channel rate.

42

Number of encoded symbols needed

Fig. 6. The number of encoded symbols needed to decode 100 input symbols, as a function of the feedback channel loss rate. The forward channel loss rate is fixed at 5%.

43Fig. 7. The number of encoded symbols needed to decode 100 input symbols at 50 receiving nodes, for various forwarded packet loss probabilities.

44

Computational load on the motes

Fig. 8. The amount of time required to decode a randomly chosen encoded packet, as a function of the number of encoded symbols already transmitted.

• 2 TelosB motes, one mote serves a single page (consisting of multiple packets) to the other mote.

45

Total number of packets transmitted

Fig. 9. The total number of packets transmitted on forward and feedback channels in order to disseminate a 5 page program to 10 motes using variants of the Deluge over-the-air programming protocol.

• 11 motes, one of which broadcast 5 pages in memory (totally 11.5K) to the 10 other motes. • All feedback channels from the 10 motes to the broadcaster were set to have a 5% packet loss rate, and the forward channel loss rates were varied from 0% to 9%.

46

Total energy used

Fig. 10. Total energy used by all the motes for communication and decoding during the dissemination of a 5 page program using a variant of the Deluge over-the-air programming protocol.

47

Conclusion

• Shifted LT codes provide an easily implemented improvement over existing rateless codes, LT codes and RT codes.

• The corresponding improvements in communication complexity, energy usage, and, in certain cases, memory requirements are even starker within a broadcast.

48

References• [3] Amos Beimel, Shlomi Dolev, and Noam Singer, “Rt oblivious erasure correcting”, IEEE/ACM

Trans. Netw., vol. 15, no. 6, pp. 1321–1332, 2007.• [4] J.W. Hui and D. Culler, “The dynamic behavior of a data dissemination protocol for network

programming at scale.”, in SenSys’04, Baltimore, Maryland, USA, Nov. 2004.• [10] A. Hagedorn, D. Starobinski, and A. Trachtenberg, “Rateless deluge: Over-the-air programming

of wireless sensor networks using random linear codes”, in IPSN ’08: Proceedings of the 7th International Conference on Information Processing in Sensor Networks, 2008.

• [11] M. Rossi, G. Zanca, L. Stabellini, R. Crepaldi, A. F. Harris, and M. Zorzi, “Synapse: A network reprogramming protocol for wireless sensor networks using fountain codes”, in SECON ’08: Proceedings of the IEEE Conference on Sensor, Mesh and Ad Hoc Communications and Networks, 2008.

• [13] S. Kokalj-Filipovic, P. Spasojevic, E. Soljanin, and R. Yates, “Arq with doped fountain decoding”, in ISSSTA 08’: International Symposium on Spread Spectrum Techniques and Applications, 2008.

• [14] S. Agarwal, A. Hagedorn, and A. Trachtenberg, “Rateless codes under partial information”, in ITA ’08: Information Theory and Applications Workshop, 2008.

• [17] Phil Levis, “Tossim: Accurate and scalable simulation of entire tinyos applications”, in In Proceedings of the First ACM Conference on Embedded Networked Sensor Systems (SenSys 2003), 2003.

• Weiyao Xiao, Sachin Agarwal, David Starobinski, Ari Trachtenberg: Reliable Wireless Broadcasting with Near-Zero Feedback. IEEE INFOCOM 2010: 2543-2551

Documents

Rateless Coding with Feedback