Iterative optimum symbol-by-symbol decoding and fast correlation attacks

3040 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 47, NO. 7, NOVEMBER 2001

Iterative Optimum Symbol-by-Symbol Decoding and FastCorrelation Attacks

Jovan Dj. Golic, Member, IEEE

Abstract—The relation between fast correlation attacks on streamciphers based on linear feedback shift registers (LFSRs) and the Hart-mann–Rudolph (HR) algorithm for optimal symbol-by-symbol decodingof linear block codes is theoretically investigated. By iterating approximateversions of the HR algorithm, novel iterative probabilistic decodingalgorithms for fast correlation attacks are developed. In particular, thesealgorithms can be applied when a large number of nonorthogonal paritychecks are used. A number of properties of the proposed iterated versionsof the HR algorithm including convergence conditions and critical biterror rates are derived.

Index Terms—Convergence, fast correlation attacks, Hartmann–Rudolph decoding, linear block codes.

I. INTRODUCTION

Fast correlation attacks on binary linear feedback shift registers(LFSRs) in keystream generators for stream cipher applications areimportant cryptanalytic techniques which are introduced in [13].The attacks exploit the bitwise correlation between the keystreamsequence and a linear combination of the LFSR sequences, whichis also an LFSR sequence. The correlation can be represented as abinary-symmetric channel (BSC) whose capacity is typically verysmall. The goal is to reconstruct the combined LFSR sequence from anobserved segment of the keystream sequence in the known-plaintextscenario. The problem is equivalent to one of decoding a truncatedcyclic linear(n; k) block code, wherek is the combined LFSR lengthandn is the observed keystream segment length. In this problemk islarge, and the ratek=n is very small in order for the correlation attackto be successful. As bothk andn� k are large, the optimum decodingminimizing the block-error rate (e.g., the minimum-distance decodingfor a time-invariant BSC and equiprobable codewords) is not feasible.

The techniques used in fast correlation attacks are based on low-weight, preferably orthogonal, parity checks and essentially reduce toiterative error-correction decoding algorithms for BSCs. They are fea-sible for largek andn � k. Note that the parity checks correspond topolynomial multiples of the combined LFSR feedback polynomial.

An iterative hard-decision decoding technique [16] based on ma-jority decision rule in fact originates from [5], where a similar tech-nique is first introduced, although based on a more sophisticated iter-ation principle, which is later called the belief propagation (BP) prin-ciple (e.g., see [11]). An iterative soft-decision decoding technique firstproposed in [13] and later improved in [14] and [17] essentially origi-nates from [2] (direct recycling (DR) principle) and [5] (BP principle),and is based on thea posterioriprobability (APP) symbol-by-symboldecoding introduced in [12] for orthogonal parity checks. This tech-nique is here referred to as the DR–APP or the BP–APP algorithmdepending on whether the DR or BP principle is employed for the it-eration process, respectively. Another iterative soft-decision decodingtechnique known as the free-energy minimization (FEM) algorithm isproposed in [10]. The FEM algorithm is here presented in a new form

Manuscript received November 9, 2000; revised May 29, 2001.The author was with the School of Electrical Engineering, University of Bel-

grade, 11001 Belgrade, Yugoslavia. He is now with the Rome CryptoDesignCenter, Gemplus, 00143 Rome, Italy (e-mail: [email protected]).

Communicated by N. I. Koblitz, Associate Editor for Complexity Theory andCryptography.

Publisher Item Identifier S 0018-9448(01)08593-5.

which is of a similar type as that of the DR–APP algorithm. Interest-ingly, this form shows that the FEM algorithm is essentially the sameas the iterative soft-decision decoding algorithm from [9].

Naturally, techniques based on soft-decision decoding are more ef-fective and more complex than techniques based on hard-decision de-coding. Also, the BP–APP algorithm is more effective and more com-plex than the DR–APP algorithm (e.g., see [3]). The DR–APP algo-rithm is shown in [1] to be somewhat more effective than the FEMalgorithm, while the complexities are similar.

The main purpose of this correspondence is to investigate theconnection between the fast correlation attacks and the binary versionof the optimum symbol-by-symbol decoding [7] known as theHartmann–Rudolph (HR) algorithm. This algorithm minimizes thesymbol-error rate, and has complexityO(n2n�k) for a binary linear(n; k) block code, since it involves a computation over all the paritychecks. Our first objective is to define an iterative version of the HRalgorithm for time-variant BSCs. We utilize the original form of thealgorithm [7] where the codewords are assumed to be equiprobable,while the channel noise probabilities are iteratively recomputed. Thisis in line with the principles of fast correlation attacks (see also [2]and [17]), but in contrast with the iterative version [8] based on theextended form of the HR algorithm where the codeword probabilities,instead of being fixed and equal, are iteratively recomputed. It isshown that the iterative version of the HR algorithm can be definedin terms of the corresponding correlation coefficients. It is importantto note that the reconstructed word obtained by the (one-step) HRalgorithm, although containing the minimum expected number oferrors, is generally not a codeword, especially if the initial number oferrors is not small. So, the main purpose of the iterative HR algorithmis to produce a codeword from this reconstructed word.

Asn� k is large in cryptanalytic applications, our second objectiveis to derive computationally feasible approximations to the iterative HRalgorithm which utilize low-weight parity checks. Unlike the iterativeAPP algorithms, the parity checks are not required to be orthogonal.This is particularly interesting for fast correlation attacks (e.g., see [6]).The DR–HR and BP–HR algorithms based on the DR and BP princi-ples are thus defined, respectively. Note that the approximation derivedin [9] is based on the DR principle, is essentially the same as the FEMalgorithm, and is different from the introduced DR–HR algorithm. Ourthird objective is to study the properties of the introduced iterative ver-sions of the HR algorithm, especially the convergence conditions for asuccessful correction of all the errors.

The iterative HR algorithm is presented in Section II and analyzed inSection III. The reduced complexity approximations to this algorithmto be used in fast correlation attacks, namely, the DR–HR and BP–HRalgorithms, are proposed in Section IV and analyzed in Section V. Forcomparison purposes, concise and unified descriptions of the iterativeAPP (DR–APP and BP–APP) algorithms and the FEM algorithm aregiven in the Appendix. Conclusions are given in Section VI.

II. I TERATIVE HR ALGORITHM

Consider a binary linear(n; k) block codeC with a parity-checkmatrixHHH to be used on a time-variant BSC with probabilities of errordefined by thea priori error probability vectorppp = (pi)

n

i=1. Thechannel is assumed to be memoryless, so that the errors are indepen-dent. In practice, a time-variant BSC results from soft-decision de-coding, wherepi is the conditional probability of error, conditionedon a received continuous channel output for theith codeword bit. If themaximuma posteriori probability (MAPP) decision rule is applied,thenpi � 0:5. For a time-invariant BSC,pi = p, 1 � i � n. A

0018–9448/01$10.00 © 2001 IEEE

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 47, NO. 7, NOVEMBER 2001 3041

time-invariant BSC results from hard-decision decoding, wherep isthe expected value of the conditional probability of error.

More precisely, leteee = (ei)ni=1 denote a random vector with inde-

pendent binary components (error bits) such thatPr(ei = 1) = pi,1 � i � n. Then, applying a random codewordxxx = (xi)

ni=1 to the

input of the BSC, we get a random received codewordyyy = (yi)ni=1 =

eee � xxx at its output, where the binary addition is componentwise. Thecodewords are assumed to be equiprobable. As usual, we keep the samenotation for a random variable and its values.

The optimum symbol-by-symbol decoding is based on deriving thea posterioriprobabilities of individual codeword symbols given a re-ceived codeword and on applying the MAPP decision rule. As such, itminimizes the probability of symbol error for each codeword symbol.A general solution to this problem for linear codes over arbitrary fi-nite fields is obtained in [7] and gives rise to a decoding rule knownas the HR algorithm. The resulting expression involves the codewordsxxx0 = (x0i)

ni=1 of the underlying(n; n � k) dual codeC 0, which is

generated by the rows of the parity-check matrixHHH .

A. A Posteriori Error Probabilities

For the BSC under consideration, instead of the codeword bits onecan equivalently deal with the error bits, so that the problem reducesto deriving thea posteriorierror probabilitiespi = Pr(ei = 1jyyy),1 � i � n. Starting from the corresponding expression from [7], aftercertain algebraic manipulations, we get

(�1)y (1� 2pi) =xxx 2C

n

j=1

((�1)y (1� 2pj))x ��

xxx 2C

n

j=1

((�1)y (1� 2pj))x

(1)

where�ji is equal to1 whenj = i and to zero otherwise. (Formally,it is assumed that00 = 1.) The denominator is proportional toPr(yyy)and is strictly positive for everyyyy such thatPr(yyy) > 0.

Another form of (1) can be obtained by introducing the correlationcoefficients and the parity-check sums. Note that this form is novelto a certain extent, and does not appear as such in [7], [2], and [8].The correlation coefficient of a binary random variableb is defined asc = Pr(b = 0)�Pr(b = 1). Letci = 1�2pi andci = 1�2pi denotethea priori anda posterioricorrelation coefficients of the random errorbit ei, respectively. Also, letxxx0 � yyy = x01y1 � � � � � x0nyn denote theparity check corresponding to a dual codewordxxx0. A parity check isthus a linear function ofyyy, and its value for a givenyyy is called theparity-check sum. Then (1) can be put into the form as shown in (2) atthe bottom of the page.

As the correlation coefficient of a binary sum of independent binaryrandom variables is equal to the product of their correlation coefficients(see [5] and [12]), the products of correlation coefficients in (2) canbe interpreted as the correlation coefficients of the binary sums of theinvolved error bits. Thus, by introducing

cxxx ; i =

n

j=1: x =1; j 6=i

cj and cxxx =

n

j=1: x =1

cj (3)

(2) can be simplified to

ci =xxx 2C :x =1

(�1)xxx �yyycxxx ; i + cixxx 2C :x =0

(�1)xxx �yyycxxx

xxx 2C

(�1)xxx �yyycxxx: (4)

Since all the parity-check sums can be obtained as linear combina-tions of the parity-check sums corresponding to the rows of the parity-check matrix, (2) implies thatccc = (ci)

ni=1 is a function ofccc = (ci)

ni=1

and the syndrome vectorHHHyyy, that is,

ccc = FFFHyHyHy(ccc): (5)

(As usual, vectors are represented as one-column matrices in matrixproducts.)

B. Time Complexity

It is interesting to determine the complexity of computing (2) forevery1 � i � n. LetMw denote the number of dual codewords inC 0

having weightw, where the weight is defined as the number of nonzeroterms. We also say that the parity check defined by a dual codewordhas the same weight as this codeword. Thus,(Mw)

nw=0 is the weight

distribution ofC 0. It is assumed thatM1 = 0, as a parity check ofweight1 would imply that the corresponding codeword bit is equal to0in all the codewords, which is impractical if not pointless. The numberof binary additions needed to compute all the parity-check sums (thatis, the signs of additive terms in (4)) is

n

w=2

Mw(w � 1) = (wav � 1)2n�k + 1 = (n� 2)2n�k�1+ 1

wherewav = n=2 is the average dual codeword weight. The numberof real additions is(n+1)(2n�k�1), and the number of real divisionsis n.

In order to obtain the minimum number of required real multiplica-tions, we need the following simple lemma.

Lemma 1: A product m

j=1aj of m real terms and the correspond-

ingm products with one term excludedmj=1: j 6=i

aj , 1 � i � m, canall be computed by3(m� 2) + 1 = 3m� 5 real multiplications.

Proof: A total of 2(m� 2) partial forward and partial backwardproducts i

j=1aj and m

j=m�i+1aj , 2 � i � m � 1, can be recur-

sively computed by2(m� 2) multiplications. The partial products areall stored and then used to compute the desired products by additionalm� 1 multiplications.

Note that the direct computation of all the products (without addi-tional storage) would require(m + 1)(m � 2) + 1 multiplications.Accordingly, computing (4) requiresn multiplications, and computingthe products in (3) can be performed byn

w=2Mw(3w� 5) multipli-

cations. This makes a total of(3n� 10)2n�k�1+n+5 real multipli-cations (provided thatM1 = 0). The time complexity is predominantlydetermined by the number of real multiplications and is thusO(n2n�k)if Lemma 1 is applied andO(n22n�k) if not. This lemma explains thecomplexity reduction which is proposed in [11] (and referred to in [3])

ci =xxx 2C :x =1

(�1)xxx �yyyn

j=1: x =1; j 6=i

cj + cixxx 2C :x =0

(�1)xxx �yyyn

j=1: x =1

cj

xxx 2C

(�1)xxx �yyyn

j=1: x =1

cj

: (2)


in a mysterious way in terms of Markov chains and the correspondingforward–backward algorithms.

C. Iterative Algorithm

The (one-step) HR algorithm consists in applying the MAPP deci-sion rule to individual error bits, that is, a received codeword bityi iscomplemented if and only ifci < 0, 1 � i � n. The reconstructedbinary word is generally not a codeword. If the number of errors is suf-ficiently small, then all or most of the errors are thus corrected and,maybe, a small number of new errors are introduced. A small numberof residual errors can possibly be corrected by some simple informa-tion set decoding technique, specific to the code in question.

However, if the number of errors is relatively large, then the numberof errors after applying the HR algorithm may just be slightly reduced.In this case, it makes sense to iteratively recompute thea posteriorierror probabilities a number of times, each time substituting the previ-ously computeda posteriorierror probabilities for the currenta priorierror probabilities. The codewords are assumed to be equiprobable ineach iteration. The error correction is performed after the last iterationstep in the same way as in the HR algorithm. The decision rule is gener-ally not optimal with respect to the probability of symbol error, that is,the reconstructed word is generally no longer optimal as a binary word,but can be much closer to a codeword. Namely, it is natural to expectthat each iteration of the iterative recomputation may effectively reducethe number of errors, so that the number of residual errors is small. Theresulting codeword, reconstructed either directly or by simple informa-tion set decoding, should approximately be optimal with respect to theprobability of block error. On a time-invariant BSC, this means that thereconstructed codeword should (ideally) be at the minimum Hammingdistance from a received codeword.

Consequently, the basic stages of the iterative HR algorithm are asfollows. Letccc(k) = (c

(k)i )ni=1 denote the vector of correlation coeffi-

cients of error bits after thekth iteration.

1. Input: received codewordyyy.

2. Precomputation: compute and store all the parity-check sums onyyy.

3. Initialization: setccc(0) = ccc.

4. Iterative update of correlation coefficients: for k = 1; 2; . . . ;kmax, compute

ccc(k) = FFFHyHyHy ccc(k�1) : (6)

5. Error correction: if c(k )i <0, then setxi=yi�1, 1� i�n.

6. Output: estimated codewordxxx = (xi)ni=1.

If the recursion (6) converges, then it must converge to a fixed pointof the (nonlinear) operatorFFFHyHyHy, and it is argued in Section III-B that afixed point is most likely composed of components taking values+1 or�1. Therefore, instead of specifying a maximum number of iterationskmax, one may use

1�

n

i=1

jc(k)i j=n < �

as a stopping criterion, where� is a fixed small real number. In the caseof success, the reconstructed codeword estimatexxx should contain lesserrors than the received codewordyyy. Note thatxxx contains no errors ifand only ifHHHxxx = 0. Residual errors may be removed by running theiterative algorithm onxxx as a modified received codeword. This can berepeated for several rounds, by usingHHHxxx = 0 or a given maximumnumber of rounds as a stopping criterion. Finally, a small number ofresidual errors, if any, may be corrected by a simple information setdecoding technique, adapted to the code under consideration. For ex-

ample, for truncated cyclic codes encountered in fast correlation at-tacks, an error-free sliding-window technique can be utilized.

III. A NALYSIS OF ITERATIVE HR ALGORITHM

A number of properties of the recursive process (6) including thecomplementation property, the fixed points, and convergence condi-tions are established in this section.

A. Complementation Property

It directly follows that the right-hand side of (1) remains unchangedif for any j, the codeword bityj is complemented, that is,yj is replacedby yj � 1, and at the same timepj is replaced by1 � pj . In terms ofthe correlation coefficients, letyyy0 be obtained by complementing somecomponents ofyyy, and letccc0 andccc0 be obtained by changing the sign ofthe same components ofccc andccc, respectively. Then

ccc = FFFHyHyHy(ccc) ) ccc0 = FFFHHHyyy (ccc0): (7)

Note that changing the sign of theith components ofccc andcccmeans thatthe ith component of the (unknown) error vectoreee is complemented,which itself means that the error for theith codeword bit is consideredwith respect to the complementedith received codeword bit.

Accordingly, the iterative HR algorithm can be modified in such away that error correction may be performed initially and at any iterationstep. For example, if the MAPP decision rule for individual error bitsis applied, then a received codeword bit is complemented if and onlyif the correlation coefficient of the corresponding error bit is negative.The sign of this correlation coefficient is then changed. This combinedoperation is called the negative correlation coefficient (NCC) error cor-rection. If such error correction is performed both initially and in everyiteration, then the correlation coefficients of all the error bits are non-negative all the time.

The resulting modified iterative process is equivalent to the orig-inal process defined by (6). Namely, (7) implies that (6) followed bythe NCC error correction, after any given number of iterations, yieldsthe same estimated codeword and the same vector of correlation coef-ficients as the modified process. Of course, the computational com-plexity of the modified process is somewhat larger, as the receivedcodeword is modified and all the parity-check sums are accordinglyrecomputed in every iteration. The advantage is that the iteratively re-computed syndrome vector, on a modified received codeword, can beused for a stopping criterion (see Corollary 1 in Section III-C).

B. Fixed Points

If the iterative process (6) converges, then it must converge to a fixedpoint of the nonlinear operatorFFFHyHyHy: [�1; 1]n ! [�1; 1]n, that is, toa vector = ( i)

ni=1 satisfying = FFFHyHyHy( ). SinceFFFHyHyHy(0) = 0, a

trivial fixed point is the zero vector0. In order to study nontrivial fixedpoints, note that (4) results into

i =Ai( ) + iBi( )

iAi( ) +Bi( )(8)

where

Ai( ) =

xxx 2C :x =1

(�1)xxx �yyy xxx ; i

and

Bi( ) =

xxx 2C :x =0

(�1)xxx �yyy xxx

(see (3)), and neitherAi( ) norBi( ) depend on i. So, a vector isa fixed point if and only if it satisfies (8) for every1 � i � n.

As iAi( ) + Bi( ) > 0, (8) is equivalent to

Ai( ) 1� 2i = 0: (9)


Therefore, ifAi( ) 6= 0, then i = 1 or i = �1, and ifAi( ) = 0,then i can take any value from[�1; 1]. Now, when a fixed point ispicked at random, sinceAi( ) is a sum of products of j and does notdepend on i, it is much more likely thatAi( ) 6= 0 thanAi( ) = 0.Further, as the zero components of are more likely to setAi( ) to 0for some values ofi than the nonzero components, ifAi( ) = 0, thenit is more likely that i = 0 than i 6= 0.

In conclusion, for most fixed points, most components are equal to1or�1, some components may be equal to0, and a very small numberof them may be different from0. Numerous experimental fast corre-lation attacks conducted by using a similar nonlinear operator in theDR–APP algorithm, described in Appendix A, are in accordance withthis conclusion.

C. Convergence in the Absence of Errors

In the case without errors, the received codeword is a codeword andthe a priori correlation coefficients for all the error bits are strictlypositive. Namely, nonnegative correlation coefficients are achieved ifhard decisions on the received codeword bits are made by applyingthe MAPP decision rule to soft channel outputs. The resulting receivedcodeword should contain no errors and zero correlation coefficients arenot allowed, as the hard decisions are then ambiguous. It is reasonableto require that the iterative HR algorithm, when applied to a receivedcodeword without errors, does not introduce errors after any numberof iterations. In mathematical terms, the iterative process (6) should besuch that the correlation coefficients inccc(k) remain strictly positive foranyk if, initially, the correlation coefficients inccc(0) are all strictly pos-itive and ifyyy contains no errors. This can indeed be proved along witha stronger property that each correlation coefficient inccc(k) tends to1whenk increases. These properties, in fact, indicate that the iterativeHR algorithm is practically meaningful.

Theorem 1: Letyyy be a codeword and let each component ofccc(0) bepositive. Then, for anyk � 0, each component ofccc(k) is positive and

limk!1

c(k)i = 1; 1 � i � n: (10)

Proof: In view of (6), (4), and (8), we obtain

c(k)i � c

(k�1)i =

Ai ccc(k�1) 1� c(k�1)i

2

c(k�1)i Ai(ccc(k�1)) +Bi(ccc(k�1))

(11)

where, asyyy is a codeword,

Ai ccc(k�1) =

xxx 2C : x =1

(�1)xxx �yyy

n

j=1: x =1; j 6=i

c(k�1)j

=

xxx 2C : x =1

n

j=1: x =1; j 6=i

c(k�1)j (12)

Bi ccc(k�1) =

xxx 2C : x =0

(�1)xxx �yyy

n

j=1: x =1

c(k�1)j

=

xxx 2C : x =0

n

j=1: x =1

c(k�1)j : (13)

We now use induction onk. Fork = 0, each component ofccc(0) isby assumption positive. For anyk � 1, if each component ofccc(k�1)

is by inductive hypothesis positive, then, according to (12) and (13),Ai(ccc

(k�1)) > 0 andBi(ccc(k�1)) > 0, respectively. From (11), we

then get thatc(k)i � c(k�1)i > 0, for every1 � i � n.

As for the second part, since the series(c(k)i )1k=0 is positive, nonde-

creasing, and upper-bounded by1, it converges to a limit i 2 (0; 1],

for each1 � i � n. Thus, the iterative process (6) converges to avector = ( i)

ni=1, and it must be a fixed point ofFFFHyHyHy. As all the

components of are positive, (12) and (13) imply thatAi( ) > 0 andBi( ) > 0, respectively. Consequently, the fixed points property fromSection III-B then implies that i = 1, 1 � i � n.

If the components ofccc(0) are nonnegative, then each component ofccc(k) is nonnegative for anyk � 0, and the iterative process convergesto a fixed point with nonnegative components. However, (10) neednot be true, and, in particular, we may have that i = 0 for somei forwhich c(0)i = 0. Then the iterative HR algorithm cannot resolve someinitial ambiguities.

Theorem 1 together with the complementation property directlyyields the following corollary, which pertains to the case where thereceived codeword may contain errors. It is a special instance of a moregeneral result from [17], but the proof given here is much simpler.

Corollary 1: If after any number of iterationsk0, the NCC errorcorrection, when applied toyyy andccc(k ), results in a codeword and inpositive correlation coefficients, respectively, then (10) remains to betrue.

We say that the iterative HR algorithm converges to a codeword ifthe iterative process (6) converges and if the resulting NCC error cor-rection yields a codeword. Corollary 1 in fact means that this happens ifand only if after a finite number of iterations the NCC error correctionyields this codeword. This is automatically checked by the modifiediterative HR algorithm, in which the NCC error correction is incorpo-rated in every iteration.

D. Convergence in the Presence of Errors

Although Corollary 1 characterizes the convergence to a codeword ofthe iterative HR algorithm, it does not provide the conditions in termsof the received codewordyyy for this to happen. Our objective in thissection is to establish such a condition it terms of the number of errorsin yyy, that is, in terms of the minimum Hamming distance betweenyyy

and a codeword. As in Section III-C, it is assumed thatyyy is obtained bymaking the hard decisions according to the NCC decision rule, so thatthe initial correlation coefficients are all nonnegative. Otherwise, theinitial number of errors makes no sense, becauseyyy may be arbitrary.The relative number of errors inyyy is called the actual bit error rate,while the expected value of the actual bit error rate (over random errorvectors) is simply called the bit error rate. Clearly, the convergence tothe nearest codeword primarily depends on the actual bit error rate, butalso on the distribution of errors, that is, on the error pattern, and onthe vector of initial correlation coefficientsccc(0). Theorem 1, althoughdealing with the case without errors, indicates that the dependence onccc(0) may be weak.

We say that the iterative HR algorithm is successful if it convergesto the nearest codeword. In general, it does not seem to be tractableto characterize allyyy andccc(0) for which the iterative HR algorithm issuccessful. Instead, what we may hope to achieve is an approximationto the maximum actual bit error rate for the iterative HR algorithm tobe successful with a high probability, when applied to a randomyyy. Wecall the approximation to be derived the critical bit error rate.

Consider the first iteration step of the iterative HR algorithm, whichessentially consists in computing (5). Since thea priori correlation co-efficients, inccc, for all the error bits are assumed to be nonnegative (i.e.,pi = (1� ci)=2 � 0:5, 1 � i � n), thea priori bit error rate is givenby

pe =1

n

n

i=1

min(pi; 1� pi) =1

n

n

i=1

pi: (14)


The minimum (Bayes) probability of error for theith bit, conditionedonHyHyHy, is given asmin(pi; 1 � pi), and the conditionala posterioribit error rate (conditioned onHyHyHy) is given by

pe(HyHyHy) =1

n

n

i=1

min(pi; 1� pi)

=1

n

n

i=1

pi +1

n

n

i=1

min(0; ci) (15)

(becauseci = 1 � 2pi).Thea posterioribit error rate is then given as the expected value of

pe(HyHyHy) over randomHyHyHy, that is, as

pe = pe +1

n

n

i=1 sss

Pr(HyHyHy = sss)min(0; ci) (16)

whereci depends onsss andccc, in view of the fact that the expected valueof pi overHyHyHy equalspi, 1 � i � n. For equiprobable codewords

Pr(HyHyHy = sss) =eee:HeHeHe=sss

Pr(eee) (17)

where

Pr(eee) =

n

i=1

pe

i (1� pi)1�e

:

It follows that0 � pe � pe � 0:5.The main point of our approach is to model the convergence of the it-

erative process (6) by the convergence of an associated iterative processdefined in terms of the bit error rates. Initially, letpi = p, 1 � i � n,wherep, 0 � p � 0:5, is the actual bit error rate inyyy. Then, thea prioribit error rate ispe = p and thea posterioribit error rate (16) reduces to

pe = p� f(p) (18)

where

f(p) =1

n

n

i=1 sss

Pr(HyHyHy = sss)max(0; �ci) (19)

because bothPr(HyHyHy = sss) andci then depend onp andsss only.More precisely, recalling thatMw stands for the number of dual

codewordsxxx0 of weightw, let for any1 � i � n, Mi; w, andM ci; w

denote the numbers ofxxx0 of weightw such thatx0i = 1 andx0i = 0,respectively(Mw = Mi; w +M c

i; w). A parity check defined byxxx0 issaid to be satisfied on a givenyyy if xxx0 � yyy = 0. Further, letmi; w(sss) andmc

i; w(sss) denote the numbers of parity checks of weightw satisfied onyyy (whereHyHyHy = sss) such thatx0i = 1 andx0i = 0, respectively, and letmw(sss) = mi; w(sss)+mc

i; w(sss). Note that forw = 0,M0 = M ci; 0 = 1

andm0(sss) = mci; 0(sss) = 1, for everyi andsss. As before, it is assumed

thatM1 = 0. Then (2) implies (20) at the bottom of the page.The associated iterative process in terms of the bit error rates is now

defined by the recursion

p(k) = p

(k�1)� f p

(k�1) (21)

for k � 1, with the initial valuep(0) = p. We study the convergenceof this process and then relate the obtained result to the convergenceof (6). First note thatf is a continuous function on[0; 0:5], such thatp � f(p) � 0 andf(0) = f(0:5) = 0. The nonnegative series

(p(k))1k=0 is hence nonincreasing and thus converges to a fixed pointof the functiong defined byg(p) = p� f(p), 0 � p � 0:5, that is, tothe largest zero� of f such that0 � � � p. In particular, ifp = 0, thenp(k) = 0, k � 0. So, for any given code, one can define the critical biterror ratep0cr such that the iterative process (21) converges to0 if andonly if p < p0cr, andp0cr is simply the minimal zero off on (0; 0:5].

Theorem 2: For any0 < p < 0:5, the iterative process (21) con-verges to0 if and only if for each0 < � � p, there existi andsss suchthatn

w=2

(1� 2�)w�2(Mi;w � 2mi;w(sss))

+

n�1

w=1

(1� 2�)w(Mci; w � 2mc

i; w(sss)) > 1: (22)

Proof: We first show that (22) is equivalent tof(�) > 0. Namely,if � > 0, then (17) implies thatPr(HyHyHy = sss) > 0 for everysss, becausethe system of linear equationsHeHeHe = sss has2k solutions andPr(eee) > 0for everyeee provided that� > 0. Consequently, it follows from (19)thatf(�) > 0 is equivalent to the condition that for at least one valueof i there exists one value ofsss such that�ci > 0. As the denominatorin (20) is strictly positive, because (20) is a special case of (2), and as1� 2� > 0, f(�) > 0 is further equivalent to (22).

Now, if (21) converges to0, thenf(p) > 0, because iff(p) = 0andp > 0, thenp(k) = p, k � 0. If f(p) > 0, then (21) convergesto the largest zero� of f such that0 � � < p. Accordingly, (21) thenconverges to0 if and only if f(�) > 0, 0 < � < p.

For a given code, thecondition (22) may not be directly checkableas itinvolvesanalyzingapolynomial in1�2�whosecoefficientsarenotnec-essarily all positive. For anyi, in order to satisfy (22),sss should be chosenso as to minimizemi; w(sss) (ideally,mi; w(sss) = 0) for small values ofw. This may not be possible for allw, because one can independentlychoose onlyn � k parity-check sums, and the remaining parity-checksums are then uniquely determined. In this case, the lowest coefficientsof the polynomial on the left-hand side of (22) will be positive and, as thecorresponding terms are dominant, we may expect that for most codes(22) will be satisfied for every0 < � � p if it is satisfied for� = p.This is checkable and in fact means thatf(p) > 0, p 2 (0; p0cr), andf(p) = 0, p 2 [p0cr; 0:5], wherep0cr is the critical bit error rate.

According to Theorem 2, given a code,p0cr is the supremum of allp,0 � p < 0:5, such that for each0 < � � p, (22) holds for somei andsss. A necessary condition for (22) to be true is that

n

w=2

(1� 2p)w�2Mi;w +

n�1

w=1

(1� 2p)wMci; w > 1: (23)

Consequently, an upper boundp0cr onp0cr is the maximum overi, 1 �i � n, of the supremum of allp, 0 � p < 0:5, such that (23) is true.Thus, ifp � p0cr, thenf(p) = 0 and the iterative process (21) remainsto be equal top in every iteration and hence cannot converge to0.

As argued at the beginning of this section, one may define the criticalbit error ratepcr for the iterative HR algorithm as the maximum actualbit error ratep such that the algorithm is very likely to be successfulwhen applied to a randomyyy with the relative number of errors equal top. The initial correlation coefficients for the algorithm are assumed tobe chosen in such a way that the correspondinga priori bit error rate

�ci =

n

w=2

(1� 2p)w�1(Mi;w � 2mi;w(sss)) +n�1

w=0

(1� 2p)w+1(M ci; w � 2mc

i; w(sss))

n

w=0

(1� 2p)w(2mw(sss)�Mw)

: (20)


pe is not smaller thanp. Namely, for any assumedpe, if the algorithmis successful forp = pe, then it will very likely be successful for anyp < pe, as the actual number of errors is then smaller. Note that inthe case of success, it may be required that the algorithm be run forseveral rounds, each time resetting the correlation coefficients to thesame initial values, where the actual number of errors is reduced ineach round.

The problem of estimatingpcr is dealt with by considering the con-vergence of the associated iterative process (21) in terms of the bit errorrates. In other words, the ability of the iterative HR algorithm to grad-ually reduce the actual bit error rate in a randomyyy may be modeledby the convergence to0 of the associated bit error rates, defined as theexpected values of the actual bit error rates over randomyyy in each itera-tion. In particular, ifp � p0cr, that is, if the iterative process (21) cannotdecrease and remains equal top in every iteration, then it is unlikelythat the iterative HR algorithm can be successful. This means thatp0crprovides an (approximate) upper bound onpcr. On the other hand, atighter approximation can be obtained by requiring that (23) be truefor most values ofi, instead of only one value ofi. If we require that(23) be true for alli, then we might even get a lower bound onpcr.

Accordingly, we propose approximatingpcr by the supremum of allp, 0 � p < 0:5, such that

Mav; 2 +

n�2

w=1

(1� 2p)w(Mav;w+2 +M cav; w)

+(1� 2p)n�1Mcav; n�1 > 1 (24)

where

Mav; w =1

n

n

i=1

Mi; w and M cav; w =

1

n

n

i=1

M ci; w: (25)

SinceM ci; n�1 � 1 and the equality can hold for only one value ofi,

it follows thatM cav; n�1 � 1=n, so that the last term on the left-hand

side of (24) is negligible. Such an approximation topcr is denoted bypHR, and can easily be numerically evaluated. It can be regarded asan important (novel) characteristic of any given binary linear code andreflects the capability of the (one-step) HR algorithm to significantlyreduce the number of errors in the received codeword. It follows from(24) thatpHR predominantly depends on the numbers of low-weightparity checks and generally increases with their increase.

IV. DR–HR AND BP–HR ALGORITHMS FORFAST CORRELATION

ATTACKS

The iterative HR algorithm is not applicable ifn � k is relativelylarge. This may happen if one chooses a linear block code with largekandn in order for the ratek=n to approach the channel capacity, e.g.,a linear code with a sparse parity-check matrix [5], [10]. Also, this oc-curs in fast correlation attacks [13] on stream ciphers based on LFSRs,as discussed in Section I, where the capacity of the corresponding BSCis typically very small. Thus, there is a need for deriving computation-ally feasible approximations to the iterative HR algorithm. This can beachieved by using numerical approximations to the expression (2) forthea posterioricorrelation coefficients of error bits, whereas the itera-tive algorithm itself can essentially remain the same (see Section II-C).The resulting iterative algorithms, apart from the main purpose ex-plained in Section II-C, have an additional objective to compensate forthe suboptimality of the expression utilized. To this end, the BP itera-tion principle is also useful.

Since (2) contains sums of products ofa priori correlation coef-ficients, which take values in[�1; 1], it follows that the computa-tional significance of a dual codewordxxx0 is inversely proportional to itsweight. Perhaps surprisingly, apart from the dual codewords involvingthe consideredith error bit(x0i = 1), the dual codewords not involving

this bit (x0i = 0) are also included. Among such dual codewords, theall-zero codeword, with the corresponding product equal to1, is dom-inant. Besides, it is natural to expect that the impact of such dual code-words onci is less significant than the impact of the dual codewordsinvolving the ith bit. Also, the denominator in (2) is independent ofi and contains the term corresponding to the all-zero codeword. Ac-cordingly, the following numerical approximation to (2) seems to besuitable

ci = ci +xxx 2�

(�1)xxx �yyy

n

j=1:x =1; j 6=i

cj (26)

where�i is a set ofxxx0, x0i = 1, with preferably low weight, and

=u= =

�1; u < �1

u; �1 � u � 1

1; u > 1

(27)

is the clipping function ensuring thatjcij � 1. For simplicity, we keepthe same notation for approximatea posterioricorrelation coefficients.Similar reasoning can be found in [2] and [9], but the obtained expres-sion (26) appears to be new, even in coding theory.

Let� = n

i=1�i denote the set of all the dual codewords used and

let ~HHH denote a parity-check matrix corresponding to�, i.e., a matrixwhose rows are linearly independent and generate�. Typically, ~HHH =HHH. Then (26) implies that

ccc = ~FFF ~HHHyyy(ccc) (28)

which is analogous to (5).It is interesting that the codewords in�i need not be orthogonal,

which is a usual assumption for the DR–APP and BP–APP algorithmsused in fast correlation attacks. Recall that a set of dual codewords(i.e., parity checks)�i is called orthogonal on theith bit if the ith bitis the only bit shared in common by the parity checks correspondingto differentxxx0 from �i. Moreover, they even need not be linearly in-dependent. The importance of linearly independent parity checks ineach�i is shown in Section V. An application of the DR–APP al-gorithm based on a large number of not necessarily orthogonal low-weight parity checks is demonstrated in [6].

Expression (26) is of a similar type as the expression (45) used in theDR–APP and BP–APP algorithms (see Appendixes A and B) and alsoas the expression (50) used in the FEM algorithm (see Appendix C).Consequently, (26) in fact shows how to use a possibly large number ofparity checks that need not be orthogonal and need not even be linearlyindependent. Moreover, even if they are orthogonal, (26) seems to bemore appropriate than (45), as it is directly rather than indirectly relatedto the exact expression (2). In addition, computing (26) requires lesstime than computing (45) or (50).

In order to determine the complexity of computing (26) for every1 � i � n, let ~Mw denote the number of codewords in� havingweightw, where ~M1 = 0 (and ~Mw = 0 if w is relatively large).The number of binary additions needed to compute all the parity-checksums needed (that is, the signs of additive terms in (26)) is

n

w=2

~Mw(w � 1) = ( ~wav � 1)j�j

where~wav is the average codeword weight in� (j�j is the cardinality of�). The number of real additions is n

i=1j�ij. The time complexity is

predominantly determined by the number of real multiplications whichis, in accordance with Lemma 1, given as

n

w=2

3(w� 2) ~Mw = 3( ~wav � 2)j�j:

If ~wav andj�j=n are upper-bounded by constants, then the time com-plexity is onlyO(n).


The iterative HR algorithm in which the approximate expression (26)is used instead of (2) is called the DR–HR algorithm. The fast corre-lation attack based on this algorithm is conceptually the same as theattack based on the DR–APP algorithm, except that (26) is used in-stead of (45), because both of them are based on the DR principle. Al-ternatively, if the BP principle is used instead of the DR principle, theDR–HR algorithm results in the approximate iterative HR algorithmwhich is referred to as the BP–HR algorithm. It is analogous to theBP–APP algorithm, but is based on (26) rather than (45).

In order to iterate (26) by using the BP principle, we have to intro-duce thea posterioricorrelation coefficientsci; �, � 2 �i, 1 � i � n,whereci; � is thea posterioricorrelation coefficient of theith error bitunder the condition that the dual codeword (parity check)� is excludedfrom �i. More precisely

ci; � = ci +xxx 2� n�

(�1)xxx �yyy

n

j=1:x =1; j 6=i

cj : (29)

This equation is used at the first iteration step only. Letc(k)i; � denote the

correlation coefficient of theith error bit after thekth iteration. Thenin the remaining steps

c(k)i; � = ci +

xxx 2� n�

(�1)xxx �yyy

n

j=1: x =1; j 6=i

c(k�1)j; xxx

: (30)

The correlation coefficientsc(k)i; � are used for the iterative process only.The error correction is based on

c(k)i = ci +

xxx 2�

(�1)xxx �yyy

n

j=1: x =1; j 6=i

c(k�1)j; xxx

: (31)

Consequently, the basic stages of the BP–HR algorithm are as fol-lows.

1. Input: received codewordyyy.

2. Precomputation: compute and store the parity-check sums(�1)xxx �yyy, xxx0 2 �.

3. Initialization: setc(0)i; � = ci, � 2 �i, 1 � i � n.

4. Iterative update of correlation coefficients: for k = 1; 2; . . . ;kmax, computec(k)i; � by (30).

5. Error correction: computec(k )i by (31) and ifc(k )

i < 0,then setxi = yi � 1, 1 � i � n.

6. Output: estimated codewordxxx = (xi)ni=1.

Although this iteration principle, later called the BP principle, is in-troduced in [5] for specific, low-density parity-check codes for bothhard-decision threshold decoding and soft-decision APP decoding (seeAppendix B), it has a more general significance, as pointed out in [11].In view of [5], [11], and [3], we may expect that the BP–HR algo-rithm has better error-correction capabilities than the DR–HR algo-rithm. Namely, the former should converge slower than the latter, andas such should make better use of the information contained in the re-ceived codeword about each error bit under consideration, especiallyin light of the fact that both algorithms are suboptimal in that they useapproximate expressions for the correlation coefficients. Note that theadvantage of the BP version over the DR version would be much lesssignificant for the iterative (optimal) HR algorithm which is based onthe exact expression (2) for the correlation coefficients.

Of course, this is achieved at the expense of somewhat increasedcomplexity of each iteration step. More precisely, the numbers of bi-nary additions and real multiplications for (26) and (30) are the same,but the number of real additions needed to compute (30) is increasedto n

i=1 j�ij(j�ij � 1). However, if an analog of Lemma 1 pertainingto real additions instead of multiplications is applied, then this numberof real additions reduces to3 n

i=1(j�ij � 1).

V. ANALYSIS OF DR–HRAND BP–HR ALGORITHMS

In this section, a number of properties of the DR–HR and BP–HRalgorithms are established. They are analogous to the properties of theiterative HR algorithm presented in Section III.

A. Complementation Property

If, for any j, the codeword bityj is complemented and, at the sametime, cj is replaced by�cj , then the right-hand side of (26) remainsunchanged forj 6= i and changes sign forj = i. The same is true for(29). Accordingly, in the same way as explained in Section III-A forthe iterative HR algorithm, the DR–HR and BP–HR algorithms can bemodified so as to incorporate error correction at any iteration step. Inparticular, the NCC error correction can be performed at any iterationstep without affecting the output of the algorithms.

B. Fixed Points

Although the DR–HR algorithm is based on (28) rather than (5), theconclusions from Section III-B regarding the fixed points of the non-linear operatorFFFHyHyHy are also true for the fixed points of~FFF ~HHHyyy. Namely,instead of (8), we now have

i = = i +Ai( )= (32)

where

Ai( ) =

xxx 2� :x =1

(�1)xxx �yyy xxx ; i:

So, if Ai( ) 6= 0, thenj ij = 1, and ifAi( ) = 0, then i can takeany value from[�1; 1]. The rest is the same as in Section III-B.

The analysis of the fixed points of the BP–HR algorithm appears tobe much more difficult.

C. Convergence in the Absence of Errors

The results derived in Section III-C for the iterative HR algorithmremain true for the DR–HR algorithm too, in particular Theorem 1 andCorollary 1. Instead of (11), the starting point now is

c(k)i = c

(k�1)i +Ai ccc(k�1) (33)

where, in the absence of errors

Ai ccc(k�1) =

xxx 2� :x =1

n

j=1: x =1; j 6=i

c(k�1)j : (34)

The proof of Theorem 1 for the DR–HR algorithm is then analogousto the proof given in Section III-C.

A theorem analogous to Theorem 1 is valid for the BP–HR algo-rithm. The only difference is that instead of (10), we have thatc

(k)i �

c(0)i , c(0)i = ci, for anyk > 0 and1 � i � n. It is proved in a similar

way as Theorem 1, first by using (30) and showing that for anyk > 0,c(k)i; � � ci, � 2 �i, 1 � i � n, and then by using (31) and showing

thatc(k)i � ci, 1 � i � n. The limit values are not determined as it ishard to analyze the fixed points of (30). However, this is not practicallyimportant as the output of the BP–HR algorithm only depends on thesign of the correlation coefficients. Consequently, both the DR–HR andBP–HR algorithms converge to a codeword if and only if after a finitenumber of iterations the NCC error correction yields this codeword.

D. Convergence in the Presence of Errors

In this section, the main lines of the convergence analysis presentedin Section III-D are adapted to deal with the DR–HR algorithm. Thebasic difference is that the DR–HR algorithm is based on the approxi-matea posterioricorrelation coefficients which are computed by (26),


rather than by the exact expression (2) used in the iterative HR algo-rithm. The main point is also to model the convergence of the DR–HRalgorithm by the convergence of an associated iterative process definedin terms of the bit error rates. This model reflects an intuitively clearfeature of iterative algorithms: the most difficult task is at the first it-eration step; if they are successful at this step in reducing the numberof errors, then, with a high probability, they will manage to correct (al-most) all the errors. Note that a similar convergence analysis of theDR–APP algorithm (see Appendix A) is conducted in [15]. That anal-ysis is supported by certain experimental evidence and turns out to bemore complicated, as it is based on (45) instead of (26).

In order to introduce the associated iterative process, consider thefirst iteration step of the DR–HR algorithm. Letci and c0i denote theapproximate and the truea posteriori correlation coefficients of theith error bit which are computed by (26) and (2), respectively. As thedecision on individual error bits is based onci rather thanc0i, the aposterioribit error rate is no longer minimal, and is given by

pe = pe +1

n

n

i=1 sss: c <0

Pr(HyHyHy = sss)c0i (35)

wherec0i depends onsss andccc, while ci depends on~sss = ~HHHyyy andccc. Tomake the analysis tractable, this expression is simplified by replacingc0iby ci. Then the associated iterative process is defined by the recursion

p(k) = p

(k�1)� f p

(k�1) (36)

for k � 1, with the initial valuep(0) = p. Here

f(p) =1

n

n

i=1 ~sss

Pr ~HHHyyy = ~sss max(0; �ci) (37)

Pr ~HHHyyy = ~sss =

eee: ~HHHeee=~sss

Pr(eee): (38)

More precisely, let~Mi; w , for any1 � i � n, denote the number ofxxx0 (parity checks) of weightw in�i and letmi; w(~sss) denote the numberof satisfied parity checks among them. It is assumed that~Mi; 1 = 0 foreveryi. Then, instead of (20), we have

�ci =

n

w=2

(1�2p)w�1 ~Mi;w�2 ~mi;w(~sss) � (1�2p) : (39)

The following theorem is then proved in a similar way as Theo-rem 2.

Theorem 3: For any0 < p < 0:5, the iterative process (36) con-verges to0 if and only if for each0 < � � p, there existi and~sss suchthat

n

w=2

(1� 2�)w�2 ~Mi; w � 2 ~mi;w(~sss) > 1: (40)

Furthermore, if the dual codewords in�i are linearly independent(e.g., orthogonal) for each1 � i � n, then Theorem 3 reduces to thefollowing theorem.

Theorem 4: If for each1 � i � n, �i is a set of linearly indepen-dent dual codewords, then for any0 < p < 0:5, the iterative process(36) converges to0 if and only if there existsi such that

n

w=2

(1� 2p)w�2 ~Mi; w > 1: (41)

Proof: If all the dual codewords in�i are linearly independent,then there existsyyy such that neither of the codewords from�i is sat-isfied. This implies that for any1 � i � n, there exists~sss such that

~mi;w(~sss) = 0 for everyw such that ~Mi; w > 0. As a consequence, foreach0 < � � p, there existi and~sss such that (40) is true if and only ifthere existsi such that

n

w=2

(1� 2�)w�2 ~Mi; w > 1: (42)

Finally, if for � = p, there existsi such that (42) is true, then forthe samei, (42) is also satisfied for every0 < � � p, because then1 � 2� � 1 � 2p. Hence, for each0 < � � p, there existsi suchthat (42) is true if and only if there existsi such that (41) is true. Thetheorem then follows from Theorem 3.

Let ~p0cr be the maximum overi, 1 � i � n, of the supremum of allp, 0 � p < 0:5, such that (41) is true. Then Theorem 4 means thatthe iterative process (36) converges to zero if and only ifp 2 [0; ~p0cr),whereas it remains to be equal top in every iteration ifp 2 [~p0cr; 0:5].

Our ultimate objective is to estimate the critical bit error rate~pcrfor the DR–HR algorithm as the maximum actual bit error ratep suchthat the algorithm is very likely to be successful when applied to arandomyyy with the relative number of errors equal top. With the samearguments as in Section III-D, one may approximate~pcr by ~p0cr or, morerealistically, by the supremum of allp, 0 � p < 0:5, such that

n

w=2

(1� 2p)w�2 ~Mav; w > 1 (43)

where

~Mav; w =1

n

n

i=1

~Mi; w: (44)

Such an approximation to~pcr is denoted by~pHR, and can easily be nu-merically evaluated. It turns out that~pHR predominantly depends on thenumbers of low-weight parity checks utilized in the DR–HR algorithm.In fast correlation attacks, where we deal with truncated cyclic codeswhose codeword length is shorter than the full period,~Mav; 2 = 0, sothat ~pHR is necessarily smaller than0:5.

Finally, it may be misleading to directly apply the same model tothe BP–HR algorithm (see [4] for the BP–APP algorithm). It is moreappropriate to view the BP–HR algorithm as a more effective way ofiterative usage of the same parity checks than the DR–HR algorithm.Accordingly, one may more conservatively regard~pHR as an approx-imate critical bit error rate for the BP–HR algorithm and as an upperbound for the DR–HR algorithm.

VI. CONCLUSION

It is well known, both in coding theory and cryptology, that iterativedecoding algorithms are very powerful in that they can be successfuleven if the error rate is much greater than half the relative minimumdistance of a code. In this paper, a number of iterative probabilistic de-coding algorithms for time-variant BSCs directly based on the HR algo-rithm for optimal symbol-by-symbol decoding of linear block codes areproposed and analyzed. They include the iterative HR algorithm, whichis based on the DR principle and on a complete set of parity checks, theDR–HR algorithm, which is based on the DR principle and on a subsetof low-weight parity checks, and the BP–HR algorithm, which is basedon the BP principle and on a subset of low-weight parity checks. In allthe algorithms, the codewords are assumed to be equiprobable, whereasthe channel noise probabilities are iteratively updated based on an ob-served codeword.

Several properties of these algorithms including the complexity, thecomplementation property, the fixed points, and the convergence condi-tions in the absence and presence of errors are theoretically derived. Inparticular, the critical bit error rates for the success of these algorithmsare introduced as novel characteristics of binary linear block codes.


It is pointed out that the DR–HR and BP–HR algorithms can beused in fast correlation attacks on stream ciphers based on LFSRs inessentially the same way as the known DR–APP, BP–APP, and FEMalgorithms. All these algorithms are presented in a unified form. Thecomplexity of the new algorithms is lower and they may particularlybe useful in the case when a large number of nonorthogonal paritychecks are utilized. In future investigations, systematic experimentsbased on computer simulations can be conducted to illustrate the de-veloped theory.

APPENDIX ADR–APP ALGORITHM

The DR–APP algorithm goes along the same lines as the iterative HRalgorithm from Section II-C, except that the correlation coefficients oferror bits are updated by a different expression. The update for eacherror bit is based on a subset of low-weight parity checks involvingthat bit in the same way as in the DR–HR algorithm from Section IV.However, instead of (26), we use

1� ci

1 + ci=

1� ci

1 + ci�xxx 2�

1�n

j=1:x =1; j 6=i

cj

1 +n

j=1: x =1; j 6=i

cj

(�1)

: (45)

Note that

ci =1� qi

1 + qi(46)

whereqi is thea posterioriprobability ratio

qi =pi

1� pi=

1� ci

1 + ci: (47)

In the error-correction stage, instead of checking if the correlation co-efficients are negative, one may equivalently check if the probabilityratios are greater than1.

Thus, (45) in fact means that thea posteriori probability ratio ofan error bit conditioned on the parity-check sums corresponding to�i is equal to the product of thea priori probability ratio of that bitand of thea posterioriprobability ratios of that bit conditioned on theparity-check sums corresponding to individual dual codewords from�i. This is true only if the parity checks corresponding to�i are orthog-onal. It is more complex to compute (45) than (26) becausen

i=1 j�ijreal multiplications rather than additions are required. Also,n real di-visions and2n real additions are required to compute (46). The addi-tional multiplications can be avoided by dealing with the logarithmsof the probability ratios, but thenn real exponentiations must be per-formed to compute (46). It is interesting that (45) reduces to (26) ifci

is close to zero for every1 � i � n.

APPENDIX BBP–APP ALGORITHM

The BP–APP algorithm goes along the same lines as the BP–HRalgorithm from Section IV, except that the correlation coefficients oferror bits are updated by different expressions, which are based on (45)rather than (26). Thus, instead of (30) and (31), we, respectively, use

1� c(k)i; �

1 + c(k)i; �

=1� ci

1 + ci�xxx 2� n�

1�n

j=1: x =1; j 6=i

c(k�1)j; xxx

1 +n

j=1: x =1; j 6=i

c(k�1)j; xxx

(�1)

(48)

and

1� c(k)i

1 + c(k)i

=1� ci

1 + ci�xxx 2�

1�n

j=1: x =1; j 6=i

c(k�1)j; xxx

1 +n

j=1: x =1; j 6=i

c(k�1)j; xxx

(�1)

:

(49)

These expressions are different from the expressions given in [11](and used in [3] and [4]), but are equivalent, and are more in line with[5], where the algorithm is originally introduced. As explained inSection IV, the BP–APP algorithm is more powerful than the DR–APPalgorithm, but the increase in complexity is here more significant.Namely, instead of n

i=1 j�ij real multiplications required to computethe products of the probability ratios in (45) for every1 � i � n, weneed three times as many real multiplications to compute the productsof the probability ratios in (48) for every� 2 �i and1 � i � n,provided that Lemma 1 is applied.

APPENDIX CFEM ALGORITHM

The iterative algorithm introduced in [10] applies to a generalproblem of reconstructing a binary vector from a noisy observationof a linear (matrix) transformation of this vector. As such, it can beused for iterative probabilistic decoding of linear block codes, such asthe codes encountered in fast correlation attacks. In particular, a goalmay be to reconstruct the binary error vector from a syndrome vector,i.e., from a vector of the parity-check sums derived from the receivedcodeword according to a given set of parity checks of preferablylow-weight. As in this case there is no noise, artificial noise must beintroduced for the algorithm to work.

The algorithm essentially consists in a gradient minimization of avariational free energy associated with the underlying probability dis-tributions in the statistical reconstruction problem considered. It is,therefore, referred to as the free-energy minimization (FEM) algorithm.In each iteration, the probabilities or error bits are updated by using spe-cific forward and backward recursions for computing certain auxiliaryprobabilities. The FEM algorithm thus seems to be quite different fromthe DR–APP and DR–HR algorithms. However, a closer examinationreveals that the recursions can be solved, thus resulting in an explicitexpression for the update of error probabilities which appears to be ofa similar type as (45) and (26), used in the DR–APP and DR–HR algo-rithms, respectively.

Namely, using the same notation as above, the FEM algorithm goesalong the same lines as the iterative HR algorithm from Section II-C,except that the correlation coefficients of error bits are updated by

ln1�c

(k)i

1+c(k)i

=ln1�ci

1+ci+�

xxx 2�

(�1)xxx �yyy

n

j=1: x =1; j 6=i

c(k�1)j

(50)

where the artificial noise is reflected in the parameter� which is slowlyincreased. Experiments conducted in [1] show that the DR–APP algo-rithm is slightly more successful than the FEM algorithm, whereas itis reported in [11] that the BP–APP algorithm outperforms the FEMalgorithm. Note that the iterative algorithm from [9] in fact coincideswith (50) for� = 1, with a minor difference that the first term on theright-hand side containsc(k�1)

i instead ofci.


REFERENCES

[1] A. Clark, J. Dj. Golic, and E. Dawson, “A comparison of fast correlationattacks,” inFast Software Encryption—Cambridge ’96 (Lecture Notesin Computer Science). Berlin, Germany: Springer-Verlag, 1996, vol.1039, , pp. 145–157.

[2] G. C. Clark, Jr. and J. B. Cain,Error-Correcting Coding for DigitalCommunications. New York: Plenum, 1982.

[3] M. P. C. Fossorier, M. J. Mihaljevic´, and H. Imai, “Reduced complexityiterative decoding of low-density parity check codes based on beliefpropagation,”IEEE Trans. Commun., vol. 47, pp. 673–680, May 1999.

[4] , “Critical noise for convergence of iterative probabilistic decodingwith belief propagation in cryptographic applications,” inAppliedAlgebra, Algebraic Algorithms, and Error Correcting Codes—AAECC13 (Lecture Notes in Computer Science). Berlin, Germany: Spril-nger-Verlag, 1999, vol. 1719, pp. 282–293.

[5] R. G. Gallager, “Low-density parity-check codes,”IRE Trans. Inform.Theory, vol. IT-8, pp. 21–28, Jan. 1962.

[6] J. Dj. Golic, M. Salmasizadeh, and E. Dawson, “Fast correlation attackson the summation generator,”J. Cryptol., vol. 13, pp. 245–262, 2000.

[7] C. R. P. Hartmann and L. D. Rudolph, “An optimum symbol-by-symboldecoding rule for linear codes,”IEEE Trans. Inform. Theory, vol. IT-22,pp. 514–517, Sept. 1976.

[8] J. Hagenauer, E. Offer, and L. Papke, “Iterative decoding of binaryblock and convolutional codes,”IEEE Trans. Inform. Theory, vol. 42,pp. 429–445, Mar. 1996.

[9] R. Lucas, M. Bossert, and M. Breitbach, “On iterative soft-decision de-coding of linear binary block codes and product codes,”IEEE J. Select.Areas Commun., vol. 16, pp. 276–296, Feb. 1998.

[10] D. J. C. MacKay, “A free energy minimization framework for inferenceproblems in modulo 2 arithmetic,” inFast Software Encryption—Leuven’94 (Lecture Notes in Computer Science). Berlin, Germany: Springer-Verlag, 1995, vol. 1008, pp. 179–195.

[11] , “Good error-correcting codes based on very sparse matrices,”IEEE Trans. Inform. Theory, vol. 45, pp. 399–431, Mar. 1999.

[12] J. L. Massey,Threshold Decoding. Cambridge, MA: MIT Press, 1963.[13] W. Meier and O. Staffelbach, “Fast correlation attacks on certain stream

ciphers,”J. Cryptol., vol. 1, pp. 159–176, 1989.[14] M. J. Mihaljevic and J. Dj. Golic´, “A comparison of cryptana-

lytic principles based on iterative error-correction,” inAdvancesin Cryptology—EUROCRYPT ’91 (Lecture Notes in ComputerScience). Berlin, Germany: Springer-Verlag, 1991, vol. 547, pp.527–531.

[15] , “A method for convergence analysis of iterative probabilistic de-coding,” IEEE Trans. Inform. Theory, vol. 46, pp. 2206–2211, Sept.2000.

[16] K. Zeng and M. Huang, “On the linear syndrome method in cryptanal-ysis,” inAdvances in Cryptology—CRYPTO ’88 (Lecture Notes in Com-puter Science). Berlin, Germany: Springer-Verlag, 1990, vol. 403, pp.469–478.

[17] M. Živkovic, “On two probabilistic decoding algorithms for binarylinear codes,”IEEE Trans. Inform. Theory, vol. 37, pp. 1707–1716,Nov. 1991.

On Computing Verdú’s Upper Bound for a Class ofMaximum-Likelihood Multiuser Detection and Sequence

Detection Problems

Wing-Kin Ma, Student Member, IEEE,Kon Max Wong, Senior Member, IEEE, and

P. C. Ching, Senior Member, IEEE

Abstract—The upper bound derived by Verdú is often used to evaluatethe bit error performance of both the maximum-likelihood (ML) sequencedetector for single-user systems and the ML multiuser detector for code-di-vision multiple-access (CDMA) systems. This upper bound, which is basedon the concept of indecomposable error vectors (IEVs), can be expensiveto compute because in general the IEVs may only be obtained using an ex-haustive search. In this correspondence, we consider the identification ofIEVs for a particular class of ML detection problems commonly encoun-tered in communications. By exploiting the properties of the IEVs for thiscase, we develop an IEV generation algorithm which has a complexity sub-stantially lower than that of the exhaustive search. We also show that forspecific communication systems, such as duobinary signaling, the expres-sions of Verdú’s upper bound can be considerably simplified.

Index Terms—Maximum-likelihood (ML) detection, multiuser detection,performance analysis, sequence detection.

I. INTRODUCTION

There are many cases in communications where we are confrontedwith the problem of detecting nonorthogonal multiple signals. Onewell-known example is code division multiple access (CDMA), inwhich the signature waveforms transmitted by different users canbe nonorthogonal to each other. On the other hand, in a single-userpulse amplitude modulation (PAM) system, the channel distortioncan destroy the orthogonality of the transmitted pulse shape, thusresulting in intersymbol interference (ISI). Maximum-likelihood (ML)detection is a powerful method of detecting nonorthogonal multiplesignals. Under the common assumption that the data symbols areindependent and identically distributed (i.i.d.), the ML detector isoptimum in the sense of minimizing the error probability [1], [2].Therefore, performance analysis of ML detection is important not onlyin studying the performance behavior of the ML detector, but also inevaluating the best achievable error rate for the respective detectionproblem.

An upper bound on the bir error probability of the ML detectorhas been derived by Verdú [3], [2]. This upper bound is based on theconcept of indecomposable error vectors (IEVs) and can provide agood approximation to the bir error probability under sufficiently highsignal-to-noise ratios (SNRs) [4]. To compute Verdú’s upper bound,it is necessary to find all of the IEVs. Unfortunately, for an arbitrarysignal correlation matrix, the IEVs may only be obtained using an ex-haustive search, which is computationally intensive for a large numberof users in the case of CDMA multiuser detection, or for a long data

Manuscript received December 21, 2000; revised June 9, 2001. This work wassupported in part under a research grant awarded by the Hong Kong ResearchGrant Council.

W.-K. Ma and P. C. Ching are with the Department of Electronic Engineering,the Chinese University of Hong Kong, Shatin, N.T., Hong Kong (e-mail: [email protected]; [email protected]).

K. M. Wong is with the Communications Research Laboratory, McMasterUniversity, Hamilton, ON L8S 4K1, Canada (e-mail: [email protected]).

Communicated by V. V. Veeravalli, Associate Editor for Detection and Esti-mation.

Publisher Item Identifier S 0018-9448(01)08970-2.

0018–9448/01$10.00 © 2001 IEEE

Documents

Iterative optimum symbol-by-symbol decoding and fast correlation attacks