© M

atth

ieu

Bloc

h 20

10

A (Very) Brief History of Coding Theory

1948-present

© M

atth

ieu

Bloc

h 20

10

Coding is Everywhere

• Magnetic recording

• Optical recording

• Wireless/wireline communication systems

• Network communications systems

© M

atth

ieu

Bloc

h 20

10

Types of Coding

• Acknowledge/Request (ARQ)

• TCP/IP: retransmit packet until received

• Forward Error Correction (FEC)

• DVD, cellular standards

• Hybrid ARQ: ARQ+FEC

© M

atth

ieu

Bloc

h 20

10

System Abstraction

“noisy”(Wireless link, Telephone line, Network link,

DVD)

Digital datavoice, music,

movie

© M

atth

ieu

Bloc

h 20

10

Coding ?

• Decoding algorithms with reasonable complexity

• Efficient encoding

• Provable properties

• Design to achieve given performance

© M

atth

ieu

Bloc

h 20

10

Gaussian Channel

message codeword coded symbol

receivedsymbol

received word

decoded message

001 0011010 1011011 001

N0

+Eb

-Eb

-Eb +Eb

k bits n bits

QPSK

© M

atth

ieu

Bloc

h 20

10

• Objective: bit error rate of 10-5

• Power efficiency:

• Spectral efficiency:

Performance of a Code

Eb

N0

ρ � R

W

© M

atth

ieu

Bloc

h 20

10

1948

Eb

N0≥ 2ρ − 1

ρ

Claude E. Shannon1916-2001

-2 0 2 4 6 8 10 120.0

0.5

1.0

1.5

2.0

Power efficiency (dB)

Sp

ect

ral

Effi

cie

ncy

(b

/2D

)

© M

atth

ieu

Bloc

h 20

10

1948Cutoff rate

“practical coding limit”

-2 0 2 4 6 8 10 120.0

0.5

1.0

1.5

2.0

Power efficiency (dB)

Sp

ect

ral

Effi

cie

ncy

(b

/2D

)©

Mat

thie

u Bl

och

2010

1948Uncoded QPSK

Benchmark for 10-5

error rate

Eb

N0=

ρ =

9.6 dB

2.0-2 0 2 4 6 8 10 12

0.0

0.5

1.0

1.5

2.0

Power efficiency (dB)

Sp

ect

ral

Effi

cie

ncy

(b

/2D

)

© M

atth

ieu

Bloc

h 20

10

1958Hamming codes

binary linear block codes

Hamming (n,k)

# message bits

# codeword bits

decreasespectral

efficiency

coding gain

-2 0 2 4 6 8 10 120.0

0.5

1.0

1.5

2.0

Power efficiency (dB)

Sp

ect

ral

Effi

cie

ncy

(b

/2D

)©

Mat

thie

u Bl

och

2010

1958Golay codes

Reed-Muller codes

-2 0 2 4 6 8 10 120.0

0.5

1.0

1.5

2.0

Power efficiency (dB)

Sp

ect

ral

Effi

cie

ncy

(b

/2D

)

© M

atth

ieu

Bloc

h 20

10

1968BCH and RS

codes

non-binary linear block codes

-2 0 2 4 6 8 10 120.0

0.5

1.0

1.5

2.0

Power efficiency (dB)

Sp

ect

ral

Effi

cie

ncy

(b

/2D

)©

Mat

thie

u Bl

och

2010

1968Soft-decoding

-2 0 2 4 6 8 10 120.0

0.5

1.0

1.5

2.0

Power efficiency (dB)

Sp

ect

ral

Effi

cie

ncy

(b

/2D

)

1969 Mariner Mission to Mars

1.78 dB

© M

atth

ieu

Bloc

h 20

10

1968Low-density Parity-Check

Convolutional codes

“Graph-based” codes

-2 0 2 4 6 8 10 120.0

0.5

1.0

1.5

2.0

Power efficiency (dB)

Sp

ect

ral

Effi

cie

ncy

(b

/2D

)

1968 Pioneer “Weather Network”

© M

atth

ieu

Bloc

h 20

10

1978Viterbi decoder

-2 0 2 4 6 8 10 120.0

0.5

1.0

1.5

2.0

Power efficiency (dB)

Sp

ect

ral

Effi

cie

ncy

(b

/2D

) Maximum-likelihood decoding on a trellis with 2m

states

© M

atth

ieu

Bloc

h 20

10

1978Concatenated

codes

-2 0 2 4 6 8 10 120.0

0.5

1.0

1.5

2.0

Power efficiency (dB)

Sp

ect

ral

Effi

cie

ncy

(b

/2D

)©

Mat

thie

u Bl

och

2010

1988Big Viterbi

Decoder (BVD)

-2 0 2 4 6 8 10 120.0

0.5

1.0

1.5

2.0

Power efficiency (dB)

Sp

ect

ral

Effi

cie

ncy

(b

/2D

)

1989 Galileo Mission to Jupiter

© M

atth

ieu

Bloc

h 20

10

1998Cellular

standards

-2 0 2 4 6 8 10 120.0

0.5

1.0

1.5

2.0

Power efficiency (dB)

Sp

ect

ral

Effi

cie

ncy

(b

/2D

)©

Mat

thie

u Bl

och

2010

1998Soft-decoding

algorithms

-2 0 2 4 6 8 10 120.0

0.5

1.0

1.5

2.0

Power efficiency (dB)

Sp

ect

ral

Effi

cie

ncy

(b

/2D

)

© M

atth

ieu

Bloc

h 20

10

1998Turbo-Codes

-2 0 2 4 6 8 10 120.0

0.5

1.0

1.5

2.0

Power efficiency (dB)

Sp

ect

ral

Effi

cie

ncy

(b

/2D

)

Iterative decoding

© M

atth

ieu

Bloc

h 20

10

1998LDPC codes

-2 0 2 4 6 8 10 120.0

0.5

1.0

1.5

2.0

Power efficiency (dB)

Sp

ect

ral

Effi

cie

ncy

(b

/2D

)

© M

atth

ieu

Bloc

h 20

10

2008Polar codes

-2 0 2 4 6 8 10 120.0

0.5

1.0

1.5

2.0

Power efficiency (dB)

Sp

ect

ral

Effi

cie

ncy

(b

/2D

)©

Mat

thie

u Bl

och

2010

What’s missing ?

• Increasing power efficiency without losing spectral efficiency

• signal set expansion

• ECE 6604 - Digital Communications

• Understanding fundamental limits

• ECE 6606 - Information Theory

© M

atth

ieu

Bloc

h 20

10

2018 ?

• Modern code designs

• Golden code, Raptor codes, Polar codes

• New applications

• network coding, quantum coding

• Still many design issues

• high-speed decoding, wireless channels