Embed Size (px)
Citation preview
1
Nearly 60 Years of Coding(and counting…)
Extremely brief introduction to coding theory
Eran Hof
2
C. E. Shannon 1916-2001
3
The creation of bits – 1948How should we count information?
2( ) ( ) log ( )X X
x
H X P x P x= −∑
“The choice of a logarithmic base corresponds to the choice of unit for measuring information. If the base 2 is used the resulting units may be called binary digits, or more briefly bits…”
quote C.E. Shannon, 1948
4
A general communication system model
C. E. Shannon, “A mathematical theory of communication,” Bell Syst. Tech. J., July 1948.
5
C. E. Shannon, “A mathematical theory of communication,” Bell Syst. Tech. J., July 1948.
6
Digital communication system model
InformationSource
SourceEncoder
ChannelEncoder
Modulator
Channel
DestinationSourceDecoder
ChannelDecoder
Demodulator
ku n
x
yu
message (k bits)
codeword (n bits)
/R k n=code rate
Digital Source
Discrete time equivalent channel
Digital Sink
7
Shannon’s coding theorem (1948)
Every channel has a capacity C, such that for any rate R<C, there exists codes of rate R that, with maximum likelihood decoding, have an arbitrarily small decoding error probability.
For R>C, reliable communication is impossible!
8
Shannon’s coding theorem (1948)
( )
,
1
122
( , )( ; ) ( , ) log
( ) ( )
max ( ; )
log 1
XYXY
x y X Y
nP
P x yI X Y P x y
P x P y
C I
C SNR
=
=
= +
∑
X
X Y
For the capacity of the additive white Gaussian noise channel weget the following
Shannon’s most celebrated result!
T. Cover and J. Thomas, Elements of Information Theory, Wiley Inter-Science, New York, 1991.
mutual information
Capacity (bits/ch. use)
bits/ch. use
9
Capacity of the ideal AWGN channel with Gaussian inputs and withequiprobable M-PAM inputs
G. D. Forney Jr., G. Ungerboeck, “Modulation and coding for linear Gaussian channels,” IEEE Trans. Inform. Theory
10
Coding gain (PAM case)Unlimited bandwidth, fixed bandwidth
22 1norm R
SNRSNR
−�
11
Error probability Vs Eb/N0Capacity limits for unconstrained AWGN
Benedetto et al. “Concatenated convolutional codes with interleavers,” IEEE Commun. Mag.
12
Informative vs. constructive results
o The theorem guarantees the existence of codes but it does not indicate how to construct these codes
o Long blocklengths are needed – lots of codewords, MLD must compute the likelihood per each codeword
o 2 major problems:o Constructing good long codes.
o Finding “easily” implementable methods for decoding
13
Meeting Shannon’s challenge
o Shannon’s result has posed a magnificent challenge to succeeding generations of researchers
o As early as the 1960’s, sequential decoding of binary convolutional codes was shown to be an implementable method for achieving the cutoff rate.
o In the bandwidth-limited regime, no practical progress beyond uncoded multilevel modulation until the invention of trellis-coded modulation in the mid-1970’s and its widespread implementation in the 1980’s
14
Meeting the Shannon’s challenge (cont.)
o Many leading communications engineers took the cutoff rate to be the “practical capacity,” and concluded that the “problem of reaching capacity had effectively been solved”
o Only in the past decade can we say that methods of approaching capacity have been found for practically all linear binary input Gaussian channels –Turbo and LDPC schemes operate within tenths of a
decibel of the Shannon limit!o For channels with strong frequency-selective fading it has long been known that multicarrier modulation can achieve the “water pouring” prescription
o Practical realizations of multicarrier modulations with powerfull codes have only appeared in recent years.
15
Water pouring
16
So you know the end, now comes the (brief) story…
17
Algebraic codeso The early days
o M. J. E. Golay codes (1949)o Richard W. Hamming (1950)o Irwin S. Reed, David E. Muller (1954)
o Cyclic Codes, Eugene Prange (1957)o BCH
o Binary BCH: Bose and Chaudhuri (1960), Hocquenghem (1959)o Cyclic structure and (the first) decoding algorithm: Peterson (1960)
o Non-binary BCH: Gorenstein and Zierler (1961), o Reed-Solomon (1960)o Berlekamp’s iterative decoding algorithm
o Majority-logic Decodable and Finite Geometry Codes (Euclidean geometry codes, projective geomtry codes, qudratic residue codes, …)
18
Linear Block CodesHamming (4,7)
1 1 0 1 0 0 0
0 1 1 0 1 0 0
1 1 1 0 0 1 0
1 0 1 0 0 0 1
(1 1 0 1)
(0 0 0 1 1 0 1)
0
1 0 0 1 0 1 1
0 1 0 1 1 1 0
0 0 1 0 1 1 1
T
T
= ⋅
=
=
=
⋅ =
=
= ⋅
x u G
G
u
x
x H
H
s y H
Richard W. Hamming
19
Hamming (4,7)Syndrome decoding
0
1 0 0 1 0 1 1
0 1 0 1 1 1 0
0 0 1 0 1 1 1
T
T
⋅ =
=
= ⋅
x H
H
s y H
S. Lin and D. J. costello Jr., Error Control Coding, 2nd ed., Prentice Hall, NJ, 2004.
20
The very early days of coding
RM(1,5) Mariner 1969
For any integer 3m ≥ , there exists a Hamming code Code length: 2 1mn = − Number of information bits: 2 1mk m= − − Number of parity-check symbols: n k m− = Error-correcting capability:
min1 ( 3)t d= =
For any integers m and r with 0 r m≤ ≤ , there exists a ( , )RM r m code Code
length: 2mn =
Number of information bits: 11 2
m m mk
r
= + + + +
�
Minimum distance min 2m r
d−=
One way of construction: For 1 i m≤ ≤ let ( )0 0,1 1,0 0, ,1 1i =v … … … �� … ( 12m i− + alternating 12i− -tuples)
{ }( , ) 0 1 2 1 2 1 3 1, , , , , , , , , ,RM r m m m mG upto r−= v v v v v v v v v v� � �
21
RM codes (simple example)
( )( )( )( )( )
( )( )( )( )
(16,11) min
0
4
3
2
1
3 4
2 4
1 4
2 3
1 3
4
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1
0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1
0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1
0 0 0 0 0 0 0 0 0 0 1 1 0 0 1 1
0 0 0 0 0 0 0 0 0 1 0 1 0 1 0 1
0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 1
RMG d =
=
=
=
=
=
=
=
=
=
=
v
v
v
v
v
v v
v v
v v
v v
v v ( )( )1 2
0 0 0 0 0 1 0 1 0 0 0 0 0 1 0 1
0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1=v v
22
RM codes with Majority-Logic Decoding
S. Lin and D. J. costello Jr., Error Control Coding, 2nd ed., Prentice Hall, NJ, 2004.
23
The very early days of coding (cont.)
[ ](24,12) 12
1 0 0 0 1 1 1 0 1 1 0 1
0 0 0 1 1 1 0 1 1 0 1 1
0 0 1 1 1 0 1 1 0 1 0 1
0 1 1 1 0 1 1 0 1 0 0 1
1 1 1 0 1 1 0 1 0 0 0 1
1 1 0 1 1 0 1 0 0 0 1 1
1 0 1 1 0 1 0 0 0 1 1 1
0 1 1 0 1 0 0 0 1 1 1 1
1 1 0 1 0 0 0 1 1 1 0 1
1 0 1 0 0 0 1 1 1 0 1 1
0 1 0 0 0 1 1 1 0 1 1 1
1 1 1 1 1 1 1 1 1 1 1 0
GolayG P I
P
=
=
NASA Deep Space (color image transmission)
US government standards for ALE
Saturn V2 1981
24
Cyclic codes
A linear code C is called a cyclic code if every cyclic shift of a codeword in C is also a codeword in C
Evariste Galois (1811-1832)
25
Binary BCH and RSFor any integers 3m ≥ and 12mt −< , there exists a binary BCH code Block length: 2 1mn = − Number of parity-check digits: n k mt− ≤ Minimum distance min 2 1d t≥ +
Let (2 )mGFα ∈ . The generator polynomial ( )g X of the t -error correcting
BCH code of length 2 1m − is the lowest-degree polynomial over (2)GF that
has 2 3 2, , , , tα α α α� as its roots
For moderate codelength, BCH codes are often the best known codes in terms of their ( , , )n k d parameters
Algebraic error-correcting decoders that are based on hard decisions are not appropriate for the AWGN channel (hard decisions cost 2-3 dB) There exist efficient soft-decision decoding algorithms (GMD, Chase,…) for BCH and other algebraic block codes,. q -ary BCH code (Let ( )mGF qα ∈ …)
Reed-Solomon codes are the special subclass of q -ary BCH codes for which 1m =
26
Error performance for (255,233,33) and (255,239,17) RS codes
Gustave Solomon and Irving S. Reed
RS codes (1960) are the outstanding practical success story of the fields of algebraic block codes
S. Lin and D. J. costello Jr., Error Control Coding, 2nd ed., Prentice Hall, NJ, 2004.
27
Convolutional codesThe workhorse of communication systems
Peter Elias
Sequential decoding (Wozencraft & Reiffen, 1961)
Threshold decoding (Massey, 1961)
Viterbi algorithm: ML decoding algorithm by A. J. Viterbi, 1967.
BCJR algorithm: MAP decoding algorithm by Bahl, Cocke, Jelinek and Raviv (1974).
+
+
A rate ½ binary nonsystematic feedforward convolutional encoder with memory order 3
28
A few more examples of convolutional encoders
+
+
+
Rate 2/3 nonsystematic feedforwrad encoder with memory order 1
+
Rate ½ binary systematic feedforward encoder with memory order 3
+
+
+
+
+
Rate 2/3 binary systematic feedback encoder with memory order 2
3( ) 1 1D D D = + + G
29
We don’t just tape bits to XORsThese things actually have structural properties!
o Algebraic structureo Polynomial representationo Matrix representation
o Linear feedforward (and feedbackward) shift register implementation.
o Finite state diagram representationo Trellis Diagramso Input Output Weight Enumerating Function
(IOWEF)o …
30
Trellis diagram
Bahl et. Al., “Optimal decoding of linear codes for minimizing symbol error rate,”
31
Viterbi algorithm (and BCJR)
o Maximum Likelihood decoding minimizes the word error for convolutional codeso Its actually a shortest path DP algorithm, BUT Viterbi didn’t know that!!o Metrics are tailored for the “channel” alphabet. Working on soft outputs is best but has
a computational cost (hard decisions or poor quantization have their coding gain cost)o The BCJR algorithm minimizes the symbol or bit error rate.o The BCJR algorithm actually tackles the more general problem of estimating the a-
posteriori probabilities (APP) of the states, and transitions of a Markov source observed through a noisy discrete memoryless channel, i.e., it generates soft decisions for symbols!
A. J. Viterbi
32
“…Nobody thought that it had any potential for practical value…”
o Published 1967 (Viterbi, though Jim Messey encouraged to include the algorithm)
o “…this decoding algorithm is clearly suboptimal….”o Optimum trellis decoder (Forney)o Standard forward dynamic programming (Omura)o First implementation (Jacobs and friends in LInkabit Corp.) “It was a big monster filling a rack”
o 1970 o “Coding is dead” (The first IEEE Comm. Theory Workshop)o VA became part of the coding standard for deep-space communication
o TCM…o VA could be used as a ML sequence detector for digital sequences in the presence of ISI and AWGN (1972 paper by Forney)
33
Few applications for VAo Deep Space Communication: Galileo mission in 1992 (JPL) 2^14-state rate ¼ convolutional code and a set of variable-strength RS outer codes - 2 dB from Shannon limit at 10^-6 (word record prior to Turbo Code)
o TCM: V.32 (1986) and V. 34 (1994), 16 and 64-state trellis codes to obtain coding gains of 4.0-4.5 dB
o Quallcomm CDMA: 2^8 state, rate 1/3 convolutional code and a VA
o GSM (TDMA): VA for both a 16-state, rate ½ convolutional code and for Equalization
VA decoders are currently used in about one billion cellphoneso General applications to hidden Markov models – most current IEEE references to the VA occur in such Transactions as Pattern and Machine Intelligence or Systems, Man and Cybernetics
34
So which is better?Algebraic codes vs. convolutional codes: the AWGN case
o Algebraic error-correcting decoders that are based on hard decisions are not appropriate for the AWGN channel (hard decisions cost 2-3 dB).
o There do exist some soft decision decoding algorithm for algebraic codes BUT…
“We know of no published results showing that the performance-complexity tradeoff for moderate-complexity binary block codes with soft-decision algebraic decoding can be better than that of moderate complexity binary
convolutional codes with trellis-based decoding on AWGN channels.”(Forney and Ungerboeck, 1998)
o Convolutional codes offer a much better performance/complexity tradeoff than block codes
o Its possible to obtain more than 6 dB of coding gain with a 32/16-state rate 1/2 / 1/3 convolutioanl code with a simple VA decoder
NO BLOCK CODE APPROACHES SUCH PERFORMANCE WITH SUCH MODEST COMPLEXITY!
o For power limited channels (Deep-Space) convolutional codes rather than block codes have been used almost from the earliest days
35
Concatenated coding schemes
o Moderate-strength “inner code” with ML decoder (or near ML) and a powerful algebraic “outer code”
o Error rate could decrease exponentially while decoding complexity increases only polynomially
o Reed-Solomon codes are ideal for use as outer codes (and we have extremely powerful algorithms for RS codes)
o RS codes are also ideal burst error correctors (counted in guard space).
G. David Forney Jr.
father of CCIT V.29, 32, 34.
G. D. Foreny Jr., Concatenated Codes, MIT Press, Cambridge, 1966.
36
Pre-1993 State of the art on the AWGN channel
Taken from Dr. I. Sason web site
37
Trellis coded modulation
o When practical coding techniques were first proposed in the late sixties, it was generally believed that coding would benefit only power-limited channels where bandwidth was plentiful
o But then power grew (or not) and bandwidth became an issue
o We must couple coding with symbols of a higher constellation level.
o If constant envelope signals are required (nonlinear channels) ,the choice must be MPSK (8,16,and more), for linear channels QAM is in order.
o In both cases optimum trellis codes for a given complexity were found by Ungerboeck (1982).
38
Motivation for TCM constellation
Ungerboeck, “Channel coding with multilevel/phase signals,” IEEE Trans. Inform. Theory
39
Motivation for TCMcapacity
Ungerboeck, “Channel coding with multilevel/phase signals,” IEEE Trans. Inform. Theory
40
Ungerboeck TCMUngerboeck, “Channel coding with multilevel/phase signals,” IEEE Trans. Inform. Theory
StateMachine
Information bits Mapping(to constellation point)
Transmitted symbol
Ungerboeck TCM
o Framework for designing the SM and Mapping lawo Its nut just exhaustive search, there is some thinking involved (partition sets, free distance,…)
o Implementation via convolutional encoder and mapping (each signal configuration seems to require a different code)
o Extensions to two-dimensional or more coding which provides both improvements and flexibility (Ungerboeck, Wei)
k n
41
Ungerboeck TCM(Coded 8-PSK 2 bits/trans.)
Ungerboeck, “Channel coding with multilevel/phase signals,” IEEE Trans. Inform. Theory
1.1 and 3.0 dB Gain over 4-PSK
42
Ungerboeck TCM(Coded 8-PSK 2 bits/trans.)
Ungerboeck, “Channel coding with multilevel/phase signals,” IEEE Trans. Inform. Theory
3.6 and 4.1 dB Gain over 4-PSK
43
BER for best Ungerboeck codes(8-PSK, 16-PSK)
Viterbi et al, “A pragmatic approach to trellis-coded modulation,” IEEE Communn. Mag.
44
Some other approaches
o Multilevel codeso Bit-interleaved coded modulation (originally developed for fading channels).
o “Pragmatic approaches”o Coding and modulations for fading channelso Coding for multidimensional constellationso Lattice codeso Coset codes
45
Pragmatic approach (Example by Viterbi et al)
Viterbi et al, “A pragmatic approach to trellis-coded modulation,” IEEE Communn. Mag.
46
TCM State of the art example: V. 32 and V. 34 Modems (1990, 1994)
Forney and Ungerboeck, “Modulation and coding for linear Gaussian channels,” IEEE Trans. Inform. TheoryITU-T Rec. V.34, “A modem operating at data signaling rates of up to 28,800 bits/s….,” 1994.CCIT Rec. V.32bis, “A duplext modem operating at data signaling rates of up to 14,400 bits/s…,” 1991.
47
1993
48
Turbo style
49
Turbo state of the art
Taken from Dr. I. Sason web site
50
The Consultative Committee for Space Data Systems (CCSDS) turbo
codes encoder
51
Current known applications of convolutional turbo codes
o CCSDS (deep space)o UMTS, CDMA 2000 (3G mobile)o TETRA Release 2o DVB-RCSo DVB-RCTo Inmarsat (M4)o Eutelsato IEEE 802.16
52
1963 – The granddaddy of them all
o Proposed by Gallager in 1963 and (almost) forgotten!!!o Rediscovered (and ‘re’forgotten) by various authorso Long LDPC codes operate way beyond the cutoff rate.o They reduce error floor and rarely have decoding errors (we just
get a decoding ‘failure’)o “codes defined on Graphs”
53
LDPC Codes
54
Meeting the challengeExample of (best known) irregular LDPC codes
Chung et al, “On the design of LDPC codes within 0.0045 dB of the Shannon limit,” IEEE Commun. Letters., Feb 2001.
55
What's the secret?
56
(extremely brief) introduction to Message Passing algorithms
o The expensive hardware solution
o Message passing algorithm
57
Message Passing algorithm for counting soldiers
This solution requires only “local communication hardware” and simple local computation (storage and addition of integers)
58
A swarm of guerillasOr why it never worked in my TIRUNOT
59
Tree of guerillas
60
Coding?
o How is counting soldiers related to coding?
o key ingredients:o Graphs
o Local computation
61
ReferencesInformation Theory
o C. E. Shannon, “A mathematical theory of communication,” Bell Syst. Tech. J., July 1948.
o R. G. Gallager, Information Theory and Reliable Communication, John Wiley, New York, 1968.
o T. Cover and J. Thomas, Elements of Information Theory, Wiley Inter-Science, New York, 1991.
Algebra
o G. Birkhoff and S. Maclane, A Survey of Modern Algebra, Macmillan, New York, 1953.o R. D. Carmichael, Introduction to the Theory of Groups of Finite Order, Ginn & Co., Boston, 1937.o A. Clark, Elements of Abstract Algebra, Dover, New York, 1984.o R. A. Dean, Classocal Abstract Algebra, Harper & Row, New York, 1990.o T. W. Hungerford, Abstract Algebra: An Introduction, 2nd ed., Saunders College Publishing, New York,
1997.
o J. E. Macfield and M. W. Maxfield, Abstract Algebra and Solution by Radicals, Dover, New York, 1992.
62
ReferencesSome good books on algebraic codes
o F. J. MacWilliams and J. J. A. Sloane, The Theory of Error-Correcting Codes, North Holland, Amsterdam, 1977.
o R. E. Blahut, Algebraic Codes for Data Transmission, Cambridge University Press, Cambridge, UK, 2003.o R. E. Blahut, Theory and Practice of Error Control Codes, Addison-Wesley, Mass. 1984.
o S. Lin and D. J. costello Jr., Error Control Coding, 2nd ed., Prentice Hall, NJ, 2004.o E. R. Berlekamp, Algebraic Coding Theory, McGraw-Hill, New York, 1964 (Rev. ed. 1984).o W. W. Peterson and E. J. Weldon Jr., Error-Correcting Codes, 2nd ed., MIT Press, Campridge, 1972.o R. J. McEliece, The Theory of Information and Coding, Addison-Welsley, Mass. 1977.
o S. A. Vanstone and P. C van Oorschot, An Introduction to Error Correcting Codes with Applications, Kluwer Academic, Boston, Mass., 1989.
o S. B. Wicker, Error Control Systems for Digital Communication and Storage, Prentice Hall, Englewood Cliffs, N.J., 1995.
Some few good books on convolutional codes
o A. Dholkia, Introduction to Convolutional Codes. Kluwer Academic, Dordrecht, The Netherlands, 1994.o L. H. C. Lee, Convolutional Coding: Fundamentals and Applications. Norwood, Mass., Artech House, 1997.o R. Johannesson and K. S. Zigangirov, Fundamentals of Convolutional Coding, IEEE Press, Piscataway, NJ
1999.
o A. J. Viterbi and J. K. Omura, Principles of Digital Communication and Coding. McGraw-Hill, New York, 1979.
63
ReferencesSome papers on algebraic codes (just the basics, mainly the original works)
o R. W. Hamming, “Error detecting and error correcting codes,” Bell Syst. Tech. J., April 1950.o M. J. E. Golay, “Notes on digital coding,” Proc. IEEE vol. 37, p. 657, June 1949.o D. E. Muller, “Applications of Boolean algebra to switching circuits design and to error detection,” IRE
Trans., pp. 6-12, September 1954.o I. S. Reed, “A class of multiple-error-correcting codes and the decoding scheme,” IRE Trans. Vol. 4, pp.38-
49, September 1954.o E. Prange, “Cyclic error-correcting codes in two symbols,” AFCRC-TN-57 103, Air Force Cambridge
Research Center, Cambridge Mass., September 1957.o A. Hocquenghem, “Codes corecteurs d’erruers,” Chiffres, 1959.o R. C. Bose and D. K. Ray-Chaudhuri, “On a class of error correcting binary group codes,” Inform. Control.,
vol. 3, pp. 68-79, March 1960.o W. W. Peterson, “Encoding and error-correction procedures for the Bose-Chaudhuri codes,” IRE Trans.
Inform. Theory, vol. 6, pp. 459-470, Septmber 1960.o D. Gorenstein and N. Zierler, “A class of cyclic linear error-correcting codes in pm symbols, “ J. Soc. Ind.
Appl. Math., vol. 9, pp. 107-214, June 1961.o I. S. Reed and G. Solomon, “Polynomial codes over certain finite fields,” J. Soc. Ind. Appl. Math., vol. 8, pp.
300-304, June 1960.o R. T. Chien, “Cyclic decoding procedure for the Bose-Chaudhuri-Hocquenghem Codes,” IEEE Trans. Inform,
Theory, vol. 10, pp. 357-363, October 1964.o G. D. Forney, “On decoding BCH codes,” IEEE Trans. Inform. Theory, vol. 11, pp. 549-557, October 1965.o E. R. Berlekamp, “On decoding binary Bose-Chaudhuri-Hocquenghem codes,” IEEE Trans. Inform. Theory,
vol. 11, pp. 577-580, October 1965.o J. L. Massey, “Step-by-step decoding of the Bose-Chaudhuri-Hocquenghem codes, IEEE Trans. Inform.
Theory, vol. 11, pp.580-585, October 1965.
64
ReferencesMore papers on algebraic codes
o J. L. Massey, “Shift-register synthesis and BCH decoding,” IEEE Trans. Inform. Theory, vol. 15, pp. 122-127, January 1969.
o H. O. Burton, “Inversionless decoding of binary BCH codes,” IEEE Trans. Inform. Theory, vol. 17, pp. 464-466, July 1971.
o Y. Sugiyama, M. Kasahara, S. Hirasawa, and T. Namekawa, “A method for solving key equation for decoding Goppa codes,” Inf. Control., vol. 27, pp. 87-99, January 1975.
o W. C. Gore, “Transmitting binary symbols with Reed-Solomon codes, “ Proc. CISS, Princton, NJ, pp. 495-497, 1973.
o R. E. Blahut, “Transform Techniques for error-control codes,” IBM J. Res. Dec., vol. 23, pp. 299-315, May 1979.
o T. Kasami and S. Lin, “On the probability of undetected error for the maximum distance separable codes,”IEEE Trans. Commun., vol. 32, pp. 998-1006, September 1984.
o I. S. Reed, “A class of multiple-error-correcting codes and the decoding scheme,” IRE Trans., vol. 4, pp. 38-49, September 1954.
o L. D. Rudolph, “A Class of majority logic decodable codes,” IEEE Trans. Inform. Theory, vol. 13, pp. 305-307, April 1967.
65
ReferencesVery short list of papers regarding TCM, convolutional and concatenated coding
o G. D. Forney Jr., “The Viterbi algorithm,” Proc. IEEE, vol. 61, pp. 268-278, 1973.o G. D. Foreny Jr., Concatenated Codes, MIT Press, Cambridge, 1966.o A. J. Viterbi, “Error bounds for convolutioanl codes and asymptotically optimum decoding algorithm, IEEE
Trans. Inform. Theory, vol. 13, pp. 260-269, 1967.o L. R. Bahl, J. Cocke, F. Jelinek, and J. Raviv, “Optimal decoding of linear codes for minimizing symbol error
rate,” IEEE Trans. Inform. Theory, vol. 20, pp. 284-287, 1974.o J. K. Wolf, “Efficient maximum-likelihood decoding of linear block codes using a Trellis, “ IEEE Trans.
Inform. Theory, vol. 24, pp. 76-80, 1978.o J. L. Massey, “Foundation and methods of channel encoding,” in Proc. ISIT, NTG-Fachberichte, Berlin, 1978.o G. D. Forney Jr., “Coset codes II: binary lattice and related codes,” IEEE Trans. Inform. Theory, vol. 34, pp.
1152-1187, 1988.o P. Elias, “Coding for noisy channels,” IRE Conv. Rec. p. 37-47, 1955.o J. M. Wozencraft and B. Reiffen, Sequenctial Decoding, MIT Press., Cambridge, 1961.o J. L. Massey, Threshold Decoding, MIT Press, Cambridge, 1963.o G. Ungerboeck and I. Csajka, “On improving data-link performance by increasing the channel alphabet and
introducing sequence coding,” in IEEE ISI, Ronneby, Sweden, June 1976.o G. Ungerboeck, “Channel coding with multilevel/phase signals,” IEEE Trans. Inform. Theory, vol. 28, pp. 55-
67, January 1982.o J. K. Omura, “On the Viterbi decoding algorithm, IEEE Trans. Inform. Theory, vol. 13, pp. 260-269, April
1967.o G. D. Forney Jr., “Convolutional codes II: maximum-likelihood decoding,” Inform. Control, vol. 25, pp. 222-
266, July 1974.o J. Hagenauer and P. Hoeher, “A Viterbi decoding algorithm with soft decision outputs and its applications, “
Proc. IEEE GLOBCOM, pp. 1680-1686, Dallas, Tex. November 1989.
66
ReferencesVery short list of papers regarding TCM, convolutional and concatenated coding (cont.):
o A. J. Viterbi, “Convolutioanl codes and their performance in communication systems,” IEEE Trans. Commun., vol. 19, pp. 751-772, October 1971.
o L. Van de Meeberg, “A tightened upper bound on the error probability of binary convolutional codes with viterbi decdoing,” IEEE Trans. Inform. Theory, vol. 20, pp. 389-391, May 1974.
o S. Hirasawa, M. Kasahara, Y. Sugiyama, and T. Namekawa, “Certain generalization of concatenated codes –exponential error bounds and decoding complexity,” IEEE Trans. Inform. Theory, vol. 26, pp. 527-534, September 1980.
o T. Kasami, T. Fujiwara, and S. Lin, “A concatenated coding scheme for error control,” IEEE Trans. Commun., vol. 34, pp. 481-488, May 1986.
o G. D. Forney Jr., R. G. Gallager, G. R. Lang, F. M. Longstaff, and S. U. Qureshi, “Efficient modulation for band-limited channels,” IEEE J. Select. Areas Commun., vol. 2, pp. 632-647, September 1984.
o G. D. Forney Jr., L. Brown, M. V. Eyuboglu, and J. L. Moran, “The V. 34 high-speed modem standard,” IEEE Commun. Mag., vol. 34, pp. 28-33, December 1996.
o G. D. Forney Jr., G. Ungerboeck, “Modulation and coding for linear Gaussian channels,” IEEE Trans. Inform. Theory, vol. 44, pp. 2384-2415, October 1998.
o A. J. Viterbi, J. K. Wolf, E. Zehavi, and R. Padovani, “A pragmatic approach to trellis-coded modulation,”IEEE Communn. Mag., pp. 11-19, July 1989.
o G. D. Forney, Jr., “Burst-correcting codes for the classic bursty channels,” IEEE Trans. Commun., vol. 19, pp. 772-781, May 1971.
67
ReferencesModern coding (just few, see further talks for more details):
o C. Berrou, A. Glavieux, and P. Thitimajshima, “Near shannon limit error correcting coding and decoding: Turbo codes,” Proc. IEEE Intl. Conf. Comuun. (ICC 93), pp. 1064-1070, Geneva, Switzerland, May 1993.
o C. Berrou and A. Glavieux, “Near optimum error correcting and decoding: Turbo-codes,” IEEE Trans. Commun., vol. 44, pp. 1261-1271, October 1996.
o S. Benedetto and G. Montorsi, “Unveiling turbo codes: some results on parallel concatenated coding,” IEEE Trans. Inform. Theory, vol. 42, pp. 409-428, March 1996.
o R. G. Gallager, Low density parity check codes, MIT Press, Cambridge, 1963.o D. J. C. Mackay and R. M. Neal, “Near shannon limit performance of low density parity check codes,”
Electron. Lett., vol. 32, pp. 1645-1646, 1996.o N. Wiberg, “Codes and decoding on general graphs,” Ph. D. Diss., Dept. of Electrical Engineering, University
of Linkoping, Linkoping, Sweeden, April 1996.o S. Benedetto, G. Montorsi, and D. Divsalar, “Concatenated convolutional codes with interleavers,” IEEE
Commun. Mag. P. 102-109, August 2003.
