Towards ideal codes: looking for new turbo code schemes

Towards ideal codes: looking for new turbo code schemes

Ph.D student: D. Kbaier Ben Ismail

Supervisor: C. Douillard

Co-supervisor: S. Kerouédan

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26th September 2011page 2/44

What is a good code?

Ph.D defense Monday 26th September 2011

Extract from «Codes and Turbo Codes» Under the direction of Claude Berrou

Dilemma: good convergence versus high Minimum Hamming Distance

Good convergence

High asymptotic gain

Ideal systemLimits to the correction capability of any codeEstablished by Shannon (1947-48)

Asymptotic gain


Turbo codes: a breakthrough in digital communications

How to combat the floor while keeping a good convergence? Turbo codes (TCs): various

communication standards (-) High floors of errors Lower error rates are required

for real-time & demanding applications

3D TCs [1] Irregular TCs [2]

Asymmetric turbo codes with different RSC encoders

Devising more sophisticated internal permutations

Component encoders with a large number of states

Different types of concatenation: serial, hybrid, multiple…

[1] C. Berrou, A. Graell i Amat, Y. Ould-Cheikh-Mouhamedou, C. Douillard, and Y. Saouter, “Adding a rate-1 third dimension to turbo codes,” in Proc. IEEE Inform. Theory Workshop, Lake Tahoe, CA, Sep. 2007, pp. 156–161.

[2] B. Frey and D. MacKay. Irregular turbocodes. In Proc. 37th Allerton Conference on Communication, Control and Computing, Illinois, page 121, September 1999.


Outline

Introduction

3-Dimensional turbo codes (3D TCs)

• 3D coding scheme

• Parameters: post-encoder, Π’ and λ

• Improving the asymptotic performance

• Improving the convergence threshold

Irregular turbo codes

Conclusion

Mrs BEN ISMAIL KBAIER Dhouha Ph.D defense Monday 26th September 2011page 5/44 Ph.D defense Monday 26th September 2011

The added part is placed just behind the pre-existing turbo encoder

λ =1/4 {1000}

3D coding scheme: encoding structure

Π

data RSC 1

X

Y1

Y2

λ Y1

λ Y2

P/S Π’PostEncoder

(1-λ) Y1

(1-λ) Y2

W

PUNCTURING

RSC 2

Classical turbo encoder

Parameters:

Permeability rate λ

Post-encoder

Permutation Π’

C. Berrou, A. Graell i Amat, Y. Ould-Cheikh-Mouhamedou, C. Douillard, and Y. Saouter, “Adding a rate-1 third dimension to turbo codes,” in Proc. IEEE Inform. Theory Workshop, Lake Tahoe, CA, Sep. 2007, pp. 156–161


Choice of the post-encoder

Influences performance in the waterfall and error floor region

Must be simple low memory RSC codes

The code is made tail biting accumulator

Must not exhibit too much error amplification

Our contribution: EXIT analysis


Post encoders EXIT analysis

k = 570 bits

λ = 1/4

R = 1/3

Max-Log-MAP10 iterationsAWGN channel


Permutation Π’

Role?

A "composite" input weight 4 square error pattern Weight of the codeword: d=28 Puncturing to R=1/2 d=16 Role of the 3D part:

• A few 1s of the redundancy part of the error pattern will be moved away to each other

• Produce a significant of additional 1s• Increasing the total codeword weight

Importance of the spread

Regular permutation

i=Π’(j)=(P0j+i0) mod P

P0=sqrt(2P)

i0~P0/2


Choice of the permeability rate λ

Convergence loss / required dmin trade-off

A large value of λ : (+) a higher dmin

(-) convergenceFER / BER

Eb/N0 (dB)

R1, λ1

R1, λ2 > λ1


Performance of 3GPP2 based 3D TCs

All simulations use the MAP algorithm with 10 decoding iterations

k = 570

R = 4/5

dmin= 4

dmin= 4

k = 3066

R= 1/3

dmin= 23

dmin= 38

dmin <= 43


Improving the asymptotic performance of 3D TCs: optimization method

All-zero iterative decoding algorithm [3] determine low weight codewords & estimate multiplicity

First terms : low multiplicity

000001000000….00100000010000.0010000000000000001000010000000001……..0001

000000000001….00000000011000000000000000..100000000010000000001000000...

000000000000000000000000000000000000000000000000000000000000000000000000

x x x x x x x x x x x x x x x x

Y1 Y1 Y1 Y1 Y1 Y1 Y1 Y1 Y1 Y1 Y1 Y1 Y1 Y1 Y1 Y1


Regular pattern

λ = 1/4Systematic part Parity y Parity w

Low weight codeword

[3] R. Garello and A. Casado, “The All-Zero Iterative Decoding Algorithm for Turbo Code Minimum Distance Computaion," IEEE International Conference on Communications, pp. 361–364, June 2004.


Improving the asymptotic performance of 3D TCs: optimization method

All-zero iterative decoding algorithm [3] determine low weight codewords & estimate multiplicity

First terms : low multiplicity Pattern of post-encoding: not regular any more

000001000000….00100000010000.0010000000000000001000010000000001……..0001

000000000001….00000000011000000000000000..100000000010000000001000000...

000000000000000000000000000000000000000000000000000000000000000000000000

Systematic part Parity y Parity w

Low weight codeword

Non regular pattern

001000100010000100001000010001000100000100010010010000100010100100100100


Optimization results for k = 1146 data bits

k = 1146

R = 2/3

λ = 1/4

Distance 12 15 21 27

Multiplicity 1 3 ≥1 ≥2

Address 1 Address 5 Address 9 Address 13

x x x x x x x x x x x x x x x x



Ones concentrated in the systematic part

The new minimum distance of the optimized 3D TC is 33 (compared to 7)


Assessment: optimization method

Yes!

Optimization method applicable for any

family of TCs• Provided that the distance spectrum has low

multiplicities at the beginning

For the 3GPP2: • Tail bits singular points in the trellis

• Tail bits cause the codewords to be truncated

But the method “cannot” be applied with

the WiMAX permutation (ARP) • Periodic distribution of the bits

• High codewords multiplicity

• Tail biting termination better distances

Can we generalize? A slight irregular post-encoding

pattern improvement in the distance

properties

Optimistic results

implement the optimization

method especially for high coding

rates


Outline

Introduction


• 3D coding scheme

• Parameters: post-encoder, Π’ and λ

• Improving the asymptotic performance

• Improving the convergence threshold


Conclusion


Improving the convergence threshold of the 3D TC

Loss in the convergence threshold (dB) for 3GPP2 3D TCs over AWGN channel:

R = 1/3 R = 1/2 R = 2/3 R = 4/5

λ =1/8 0.15 0.13 0.06 0.01

λ =1/4 0.26 0.22 0.18 0.14

Reducing the convergence loss of 3D TCs:• Costello [4] Time Varying (TV) post-encoder• Specific Gray mapping for 3D TCs associated with high order constellations

0.19

Rayleigh channel

λ R

[4] D. Costello Jr. Free distance bounds for convolutional codes. IEEE Transactions on Information Theory, 20(3):356-365, May 1974.


Input 47

Recursivity : polynomial 5 4-state post-encoder with time-varying parity

construction (5, 4:7)

Convergence/distance trade-off

Reducing the convergence loss of 3D TCs: time varying post encoder

4-state post-encoder with time-varying parity construction (5, 4:7)

Replace periodically some redundancies W1=4

by W2=7 BER out = 2* (BER in +ξ)

(5,4:7) Distance = 2 (5,4) Distance =3 and (5,7) Distance =5

timeW1(4)

W2(7)

W2(7)

W1(4)

W2(7)

W1(4)

Replacement period L

W1(4)

W2(7)W2(7)

Time varying trellis


General results for the time varying technique

Loss of convergence reduced by 10% to 50% of the value expressed in dB

The asymptotic performance is not degraded

• For a fixed code memory, the choice of the post-encoder does not influence dmin of the 3D TC

• Higher local minimum distance of the post-encoder =

• Better level of the extrinsic information which the predecoder supplies to the two SISO decoders

The TV technique acts as a convergence accelerator of the 3D TC


Error rate performance example of time varying 3D TCs

k = 1146 bits

Loss of convergence reduced by 35% from 0.23 dB to 0.15 dB

Max-Log-MAP

10 iterations


3D TCs for high spectral efficiency transmissions

BICM approach Among the bits forming a symbol in M-QAM or M-PSK

modulations, the average probability of error is not the same for all the bits

Three constellation mappings:Configuration 1: mapping uniformly distributedConfiguration 2: systematic bits mapped to better

protected places as a priorityConfiguration 3: systematic bits (then if possible) post-

encoded parity bits protected as a priority


Example: 3D TCs associated with a 16-QAM modulator

Systematic bits &

post-encoded parity bits mapped to better protected places

1867 16-QAM symbols

4 bits of a 16-QAM symbol

2298 x

2298 y1

2298 y2

574 w

k = 2298 bits

R = 1/3

λ = 1/8

Gaussian channel

Gain: 0.22


Design rules

Configuration 1 loss of convergence still observed

Configuration 2 or 3 gain in the waterfall region Configuration 3 must be used as far as possible Otherwise, implement at least the configuration 2 Significant gain: • Even for transmissions over Rayleigh fading

channels• Increases with the coding rate R for the same λ


Properties of 3-Dimensional turbo codes

Increase in dmin

But:

• Loss in the convergence threshold

• Increase in complexity

- Why?

- The answer is in the decoding process


What about the 3D decoding complexity?

8-stateSISODEC1

8-stateSISODEC2

4-stateSISO

PRE-DECΠ’-1 S/P

Π

P/SΠ’

w

y2

Extrinsic information about the post-encoded parity bits

Π

Π-1

y1

Classical

Turbo

Decoder


Complexity figures

High throughputs

# Proc increases additional complexity decreases

k = 1530 bits

λ = 1/8

R = 1/2


Summary: 3D TCs (1/2)

BER/FER

Eb/N0 (dB)

Classical TC

3D TC

Time varying

3D TCs

+

high order modulations

+

specific Gray mapping

Optimization method

Time varying post-

encoder (5, 4:7) with a

little irregularity

Irregularity in the

Gray mapping for 3D

TCs associated with


Non regular post-

encoding pattern to

improve the asymptotic

performance


Summary: 3D TCs (2/2)

Irregularity

The next step of the study concerns the investigation of irregular TCs Why? Obtain an irregular TC which performs well in both the waterfall and

the error floor regions

Work on irregular LDPC codes significant gain

Frey & MacKay introduced irregularity to TCs

Sawaya & Boutros lower the floor of irregular TCs

Time varying post-

encoder (5, 4:7) with a

little irregularity

Irregularity in the

Gray mapping for 3D

TCs associated with


Non regular post-

encoding pattern to

improve the asymptotic

performance


Outline

Introduction



• Basics of irregular TCs

• Selecting the degree profileEXIT diagrams

• Design of suitable permutations for irregular TCsPrinciple & simulation results

• Irregular TCs with post-encoding

Conclusion


Self-concatenated turbo encoder

Equivalent encoding structure for a regular turbo encoder:• Merge two trellis encoders• double size interleaver + 2-fold repetition

Interest: introduce an irregular structure


Irregular turbo encoder

Repetition (dj) Interleaver RSC

Information bits

Repetition (d2)

Repetition (dmax)

k infobits

k f2

k fmax

k fj

1max

2

d

ddf

d

ddfdd

max

2

Degree profile (2, 3,…, dmax) or (f2, f3,…, fmax)

Parity bits

Two non-zero fractions: d =2 and d >2 :

•f2 + fmax=1•2 f2 + dmaxfmax = dAverage

Only three parameters

Performance of an irregular TC strongly depends on the degree profileNumber of degrees and fractions: 2(dmax-1)Only two equations to optimize all these parameters!


What is a good irregular turbo code?

Our approach = we separate the problems

Π

Degree profile

RSC code

It depends on:

1. Search for a good degree profile using a random interleaver

2. Optimize the interleaver

Fixed

Our contribution: analyzing the degree profile using hierarchical EXIT charts


Analyzing the degree profile using hierarchical EXIT charts


Performance example of irregular TCs

Interleaver length: 3438

dav = 3R = ¼MAP 8 iterations

k = 1146 bits


Outline

Introduction







Conclusion


Proposed algorithm for the permutation design (1/2)

Reduce the correlation effect between the pilot groups while improving the distance properties of irregular TCs

Information sequence: 0 1 0 1 0 0 1 0 1 1 0 ...

00000000 11 00 11 00 00 11111111 00 …

Appropriate repetition

weight 11 2 3 4

OriginalAddress =565

Copy 2Address =273

weight 0

Copy 3 / Address =120

Interleaver size: 576The Dijkstra’ s algorithm [5]:[5] E. Dijkstra. A note on two problems in connexion with graphs. Numerische mathematik, 1(1):269-271, 1959.


Proposed algorithm for the permutation design (2/2)

Reduce the correlation effect between the pilot groups while improving the distance properties of irregular TCs

Information sequence: 0 1 0 1 0 0 1 0 1 1 0 ...

00000000 11 00 11 00 00 11111111 00 …

Appropriate repetition

weight 11 2 3 4


Copy 2Address =273

weight 0



Copy 5Address =356

Copy 4Address =440

Copy 2Address =273

Copy 6 Address =189

Copy 6Address =500

Copy 7Address =47


weight = 0

weight = 1

Address =1

In the example: d = 8


Error rate performance of irregular TCs with an optimized interleaver


R = 1/4

Interleaver size: 144

Gain: 2.5 decades


Gain:

3.5 decades


Error rate performance of irregular TCs with an optimized interleaver

Proposed algorithm: very fast for short block sizes

For medium sizes and large blocks:

• Unacceptable computational time

• Uncertainty about detecting all the possible cases

Drawback: Necessity to store all the interleaved addresses

Devising good interleavers for irregular TCs proves to be a difficult task


R = 1/4


Gain:

> 2 decades

CPU:

Two quad core processors

(Xéon)

RAM: 8Go


Outline

Introduction







Conclusion


Adding a post-encoder to irregular TCs

We propose an irregular TC inspired by our work about 3D TCs

• Ensure large asymptotic gain at very low error rates

• Even with non optimized internal permutation

• Improve the distance properties of irregular TCs

Non-uniform

repetitionΠ RSC

Information bits

Parity bits

λ

Π’Post-encoder

1-λ


Performance example of irregular TCs with post-encoding

All simulations use the MAP algorithm with 10 decoding iterations Degree profile (f2,f8), dav = 3, R = 1/4 , λ = 1/8 and k = 4096 bits 3GPP2 interleaver, interleaver size: 12282

Gain:

2.5 decadesdmin= 33

dmin= 44

dmin= 50


Summary: irregular TCsBER/FER

Eb/N0 (dB)

Classical TC

Irregular TC

Suitable permutations

Irregular TC

+

Post-encoder


Conclusion

Towards ideal codes?

3D TCs

Asymptotic performance:

The 3D TC significantly improves performance in the error floor region

Convergence:

We can implement methods which reduce significantly the loss of convergence

Irregular TCs

Performance:

Closer to capacity but very poor asymptotic performance

Improve the distance properties:

Graph-based permutations (Dijkstra's algorithm + estimation of the minimum distance)

Irregular TCs + post-encoder


Perspectives

Towards ideal codes? 3D TCs:

• New structures

• Diversity techniques: MIMO, rotated constellations…

• Double binary

• Hardware implementation complexity of 3D turbo decoder

Irregular TCs:

• Post-encoding pattern

• The design of suitable permutations for irregular TCs is an important future research work

- Eliminate the interleavers producing low minimum distances early in the search process

Reduce the space of search

Promising algorithm even for large blocks


Thank you for your attention

Contributions to the literature

Conference papers:

1. KBAIER BEN ISMAIL Dhouha, DOUILLARD Catherine and KEROUÉDAN Sylvie, "Improving 3-dimensional turbo codes using 3GPP2 interleavers", ComNet'09: 1st International Conference on Communications and Networking, 03-06 November 2009, Hammamet, Tunisia, 2009.

2. KBAIER BEN ISMAIL Dhouha, DOUILLARD Catherine and KEROUÉDAN Sylvie, "Reducing the convergence loss of 3-dimensional turbo codes", 6th International Symposium on Turbo Codes & Iterative Information Processing, 06-10 September 2010, France, pp. 146-150.

Journal papers:

3. KBAIER BEN ISMAIL Dhouha, DOUILLARD Catherine and KEROUÉDAN Sylvie, "Analysis of 3-dimensional turbo codes", Annals of Telecommunications, available online at http://www.springerlink.com/content/1r8785617q48n106/

4. KBAIER BEN ISMAIL Dhouha, DOUILLARD Catherine and KEROUÉDAN Sylvie, "Design of suitable permutations for irregular turbo codes", Electronics Letters, June 2011, vol. 47, n° 13, pp. 748-749.

5. KBAIER BEN ISMAIL Dhouha, DOUILLARD Catherine and KEROUÉDAN Sylvie, "Improving irregular turbo codes", Electronics Letters, to appear.

Submitted journal paper:

6. KBAIER BEN ISMAIL Dhouha, DOUILLARD Catherine and KEROUÉDAN Sylvie, "Improving 3GPP2 3-dimensional turbo codes and aspects of irregular turbo codes", submitted to EURASIP Journal on Wireless Communications and Networking.

Documents

Towards ideal codes: looking for new turbo code schemes