lF.FF TRANSA<'TIONS O N INFORMATION TIIF-ORY. VOI . 36. N O . 4, JU1.Y 1990 763

The Method of Poles: A Coding Method for Constrained Channels

MARIE-PIERRE BEAL

Abstract -A method for solving coding problems involving channels restricted by finite state constraints is presented. The method was introduced by P. A. Franaszek and is based on the computation of a set of principal states. It is proven that state-independent decoding is guaranteed. It is also related to recent work in symbolic dynamics by R. L. Adler, B. Marcus and others. Typical applications are coding methods for magnetic storage.

I. INTRODUCTION

ECENTLY, A NUMBER of authors have consid- R ered the problem of encoding digital data into a constrained set of available sequences (cf. [ 11-[51, [12]-[15], [20]). The authors were motivated by physical problems that lead to limitations of transmission or stor- age systems. Typical constraints are the [d, k] constraints using in magnetic storage. The [ d , k ] words are finite binary sequences in which the number of zeros occurring between two consecutive ones has both an upper bound k and a lower bound d, where d and k are integers. When the symbol one corresponds to a magnetic transition, the lower bound controls the interference between two mag- netic transitions and the upper bound is introduced for a good synchronization of a self-clocking system. The prob- lem is then to encode an arbitrary sequence of zeros and ones into a constrained sequence. The word to be en- coded is read from left to right and the coding is sequen- tial. The decoding has to be local to guarantee that 'a single error will not propagate into the decoded word. Such a coding scheme is said to be without error-propa- gat ion.

The constrained sets of sequences can be described by the mathematical model of symbolic dynamical systems. In the case of this example, the system is a subshift of finite type, also called a finite memory constrained sys- tem, that is, it is characterized by a finite collection of finite forbidden sequences. Recent work in symbolic dy- namics has produced solutions for these coding problems (cf. [1]-[4], [17]-[20]). A previous work of P. A. Franaszek has led to different algorithms ([121-[15]).

For channels described by a subshift of finite type, P. A. Franaszek has given a method based on the compu- tation of a set of principal states 1121. Our result shows that, with a good natural choice of codewords, Franaszek's

Manuscript received March 3, 1989; revised October 20, 1989. The author is with LITP, Universitt Paris 7, 2 place Jussieu, 75251,

IEEE Log Number 9035260. Paris Cedex 05, France.

method yields codes with sliding block decoders. We explain in detail the differences between this algorithm and another algorithm of P. A. Franaszek [15] which also guarantees the limited error-propagation property. The point of view taken in this paper is that of finite automata theory. In a previous paper [5], we have studied some correspondences between notions of symbolic dynamics and notions of the theory of automata and codes. In particular, we have given in [5] a new proof of the main theorem of R. L. Adler, D. Coppersmith, and M. Hassner presented in [2], for the weaker case of strict inequality between entropies.

We present Franaszeks algorithm by constructing a finite transducer that realizes coding and decoding. Good properties of the transducer guarantee good properties of the coding. Complete and effective proofs are given. The usual coding parameters, namely the coding cost (number of output symbols over the number of input ones) and the size of buffers necessary to encode and decode, are con- sidered. The method cannot be extended to the more general case of coding symbols of a finite alphabet A of cardinality k into a subshift of finite type of entropy equal to logtk). We have called the method, the method of poles, the poles being some particular important states of the transducer.

We present in Section I1 basic notions of finite au- tomata theory and symbolic dynamics. The coding method is exposed in Section 111. The main result is contained in Proposition 5 where we prove that the transducer con- structed by our method is a local automaton in its output. Section IV is composed of numerous examples. Most of them correspond to the [ d, k ] constraints, but the last one corresponds to another kind of constraint found in some transmission systems. We finally mention without a com- plete exposition some possibilities of new optimizations. The last example shows that one should perhaps not always use blindly general algorithms and rely on theory.

11. DYNAMICAL SYSTEMS AND FINITE AUTOMATA

We provide some definitions and basic results that will be used in the following section.

A. Subshifts

Let A be a finite set which we call an alphabet. The set of all words on the alphabet A is denoted by A* and is

001 8-9448/90/0700-0763$0 1 .OO 0 1990 IEEE

764 IEEE TRANSACTIONS O N INFOKMATION TIiEOKY. VOI,. 36, N O . 4, JU1.Y 1990

equipped with the operation defined by the concatenation of two sequences. The monoid A* is called the free monoid on A . As is customary, A Z denotes the set of two-sided infinite sequences of elements of A, and AN denotes the set of corresponding one-sided sequences. We call the sequences of AZ bi-infinite words. The space AZ is endowed with the usual metric, which is defined by the following distance d

4 ( % ) , € Z # % ) , € Z ) = 2-k where k = sup { i 2 O / a - , a - , - . . . aoal

a, = b- ,b - , - I . . . bob , . . . b , ) .

In this metric the more two sequences consecutively agree the closer they are. The space AZ is compact in this topology.

We define a transformation (T of A Z onto itself by

4 % 7 ) , € Z ) = ( % + l ) , E Z

and refer to it as a shift. A subset S of AZ is said to be shift-invariant if u(S) = S. Any closed and shift-invariant subset of A' is called a subshift. A subshift that is characterized by a finite list of forbidden finite sequences is called a subshift offinite type. This means that there is a finite collection C of finite words in the alphabet A such that a bi-infinite word w belongs to the subshift if and only if no finite factor (or block) of w belongs to the collection C.

B. Finite Automata

A finite state automaton on the alphabet A is com- posed of two finite sets, Q (the set of states) and F (the set of edges). The set of edges is included in Q X A X Q. A finite path is a sequence of consecutive edges ~ ~ q , , a , , q , + l ~ ~ l s , ~ , , the words a l a 2 . - . a, being the label of this path. A bi-infinite path is an infinite sequence of consecutive edges ((q,, a,, q,+ I)), E z, the bi-infinite word (a,) , being the label of this path. Such an automaton is denoted by (Q, F ) .

A subshift is called a rational subshift or a sofic system if there exists a finite automaton ( Q , F ) such that a bi-infinite word w belongs to the subshift if and only if the word is the label of a bi-infinite path of the automa- ton. The subshift is said to be recognized by the automa- ton ( Q , F ) . Every subshift of finite type is a sofic system.

Examples: The automaton d is ( Q , F ) where Q = (1,2) and F = ( ~ l , a , l ~ , ( l , a , 2 ) , ( 2 , b , l)} (Fig. 1). It defines a sofic system S. A bi-infinite word w belongs to S if there is a bi-infinite path of ( Q , F ) labeled by w. The automa- ton 93' (Fig. 2) defines a sofic system T. The subshift S is

a

b

Fig. 1 . Subshift S of finite type.

&$-- a

Fig. 2. Rational subshift T.

a subshift of finite type but T is not. A collection of finite forbidden sequences that characterize S is reduced to the word bb.

A finite automaton (Q, F ) is deterministic if for each p in Q and a in A there is at most one state q in Q such that ( p , a, q ) is an edge of F . We denote by p ' a the state q when the edge ( p , a, q ) exists.

A finite automaton (Q, F ) is said to be transitiu if for all states p and q, there exists a path from p to q.

A finite automaton (Q , F ) is said to be right-closing if there exists an integer n such that two paths

origin (pl = q l ) and the same label also satisfy p 2 = qz. A deterministic automaton is right-closing.

A finite automaton is local if there exists two integers n and d I n such that if two finite paths ( p , , a , ,

label w = a l a 2 . . . a, then pd = p i . A transitive and local automaton is right-closing, as the

reader may check directly. Examples: The automaton d of Fig. 1 is local but the

automaton 93' of Fig. 2 is not. The following proposition characterizes local automata

in an effective way. We call a cycle in an automaton a

Proposition 1: Let d be a transitive finite automaton. The automaton d is local iff there does not exist two cycles with the same label.

( P , 4 1 ? P , + l ) 1 5 i s n and (q,?ai?q,+I)l5,<, with the Same

P,+ l )os , sn - l and ( P ; ? ~ , J 4 + l ) o < f a 7 - l have the Same

path (P, ,ar ,P,+l) ls ,s .with P I = P , .

Proofi Let us consider two cycles both labeled by the

We can assume that p , # q l . The two paths will be represented by

Same word w: ( P , 9 a , d , + l ) 1 5 , s , and ( q , d , 4 + l ) , s , s f 7 .

p---w--'p and q*q.

Let us consider two integers n and d I n such that the local property is satisfied. Let r be an integer such that the length of w r is superior to n. The definition of local automata implies that the state reached after reading w' should be the same for all paths labeled by w2'. This contradicts the fact that p and q are different.

We suppose that the automaton d is not local. For all integers n and d I n there exists a word w of length n and two paths labeled by w represented as

PO-PI-PZ"' - P f 7 , W I w2 wt,

w2 w,, W

P i ) - - ! + P i - P ~ * . .

where w, are letters, w = w,wz . . . w,, and where p,, # p;,. The integer d is now fixed equal to n/2. As the graph of this automaton is strongly connected, we cannot find two integers i and j where i I d I j and p, = p; and p, = p;.

765 &AL: METHOD OF POLES, CODING METHOD FOR CONSTRAINED CHANNELS

For large enough n we can find at least (card + 1 pairs(p;,p,!) where pi # p,'. Two of them are equal and we get two equal labeled cycles.

Let us now suppose that the input automaton is local. Let a be the relation included in AZ X B Z and defined by ( x , y) E a if ( x , y ) is the label of a bi-infinite path of

I

the transducer. This relation is a continuous function with the shift (+, that is a morphism in the

terminology of symbolic dynamics. In terms of coding, it that the function thus defined from A~ to B~ is

characterized by local conditions and, therefore, does not allow the propagation of errors.

The following result relates local automata to subshifts of finite type.

Proposition 2: A sofic system is of finite type iff it can be recognized by a local automaton. Moreover such an automaton can be chosen deterministicly.

Proofi Let S be a subshift of finite type, and let C be a finite set of finite words such that a bi-infinite word w belongs to S if and only if no finite factor of w belongs to C. Let r be the length of the longest words of C. Let d, = (Q , R), the automaton defined by Q = { a l a 2 . . . a, E

A * , a i = A } and F={(a1a2.~.a,,a,a2...a,a),ai~ A,a E A}. Let Q' be the set of states of Q such that at least one factor of a l a 2 . . . a, belongs to C , and let F' be the set of edges (q,a,q') of F such that q or q' belongs to Q'. Then the automaton (Q - Q', F - F') recognizes the sub- shift S and is deterministic.

Conversely let d be a local automaton which recog- nizes a subshift S. There exist two integers n and d I n such that, if two finite paths ( p i , a j , p i + , ) o s i ~ n - I and ( p / , ~ ~ , p / + , ) ~ ~ ~ ~ ~ , - ~ have thesamelabel w = a l a 2 ~ - ~ a , , then pd=prj. Let C be the set of words of length less than n + 1 that are not labels of a path of the automaton d. Let w be a bi-infinite word which has no factor in C. Let a l a 2 . * . a,+2 be a factor of length n + 2 of w. The words a l a 2 . . . a,+, and a2 ' . . do not belong to C . There exists at least one path of d labeled by a l a 2 a,+, and all such paths reach the same state pd+l after reading ad+l . There exists at least one path labeled by a 2 - . . an+2 and all such paths reach the same state pi after reading a d + ,. As all paths labeled by a2 . . . a,+ , reach the same state after reading the letter ad+, , we get p d + , = pi and a finite path of d labeled by a l a 2 - an+2. We get a construction of a bi-infinite path labeled by w , which proves that the subshift S is of finite type, charac- terized by the collection C.

D. Entropy

A (formal) language L is a subset of A* where A is a finite alphabet. Let L be an infinite language. The en- tropy of L is denoted by h(L) and defined by

1 n - m n

h( L ) = lim sup - log( card ( L n A " ) ) .

We get 0 I h( L ) I log(card( A ) ) .

To any subshift S we may associate a language L by considering the set of finite factors (or blocks) appearing in the bi-infinite words in S. We define the entropy of S to be the entropy of L.

Remark: The previous definition of the entropy of a language L depends on the alphabet A . Let us consider the language L' (resp. A') of the words of L (resp. A*) of length multiple of an integer q. The entropy of the language L' seen in the alphabet A' is h(L') = qh(L).

To an n-state automaton d we associate an n X n matrix M ( d ) defined as follows: the entry is the number of edges going from the state i to the state j .

The following result provides a useful characterization of the entropy.

Proposition 3: Let S be a sofic system recognized by an automaton d that is transitive and right-closing. Then h ( S ) = log(A) where A is the largest eigenvalue of the matrix M ( d ) .

Proofi As M ( d ) is a positive matrix, we have

C. Finite Transducers A = lim 11 M ( d), Ill'", for n + CQ

We will use later finite transducers for coding and decoding. A finite transducer is a finite automaton whose edges are labeled in A X B where A and B are two finite alphabets. If (p , (a ,b) ,q) is an edge of a transducer (Q,F), the letter a is called the input label and b the output label. The transducers considered in this paper will be assumed to satisfy the following property: let ( p , ( a , b ) , q ) and (p,(c,d),q) be two edges with a and c in A , b, and d in B and p and q two states. Then a = c iff b = d . This property allows us to consider the associ- ated input and output automata defined as follows. The input automaton is (9, F , ) where E', = {(p, a, q ) with p , q E Q, a E A such that there exist b E B and (p,(a ,b) ,q) E F}. The output automaton is ( Q , F 2 ) where F2 =

{(p, b, q) with p, q E Q, b E B such that there exist a E A and ( P,(Q, b), q ) E F).

where

IlNll = c c l4,l I s i s n l i j s n

for every n x n matrix N . 0

If d = (Q, F ) and F = (f,; . .,f,,}, we define the automa- ton A? on the alphabet F by @=(Q,F ' ) such that if (q, a;, q') = f; is an edge of F then (q,f;, q') is an edge of F'. One cannot find two equal labeled edges in the automaton A?, it recognizes a subshift S'. Let L (resp. L') be the language formed by factors of words of S (resp. S'). We have card(L'n F " ) = IIM(d)'*ll and h(L') = log(A).

Let 4 be a state of d. Let L, be the set of words that are labels of paths of d going from q to q. We have L, c L . Let m be the number of states of d. For every

766 IEEE TKANSAC'TIONS ON INFORMATION TIIFOKY, vel,. 36, NO. 4, J U I . Y 1990

word w in L, there exists U, L I E A* of length less than m such that uwu belongs to L,. It follows that h ( L J = h(L) . We also have h(L;) = h(L') where L; is the set of words that are labels of paths of Si9 going from q to q. Since the automata Q? and Si9 are right-closing, in each automaton, two equal labeled cycles going from p to p are equal. We get h(L,) = h(L;) and as a consequence h ( L ) = h(L') =

log (A). Remarks:

Let S be a sofic system recognized by a nontransis- tive automaton Q?. The entropy of S is the maxi- mum of the entropies of all sofic systems recognized by the strongly connected components of the au- tomaton (cf. [41 3.23). Proposition 3 is still true when Q? is a transitive nonambiguous automaton. A finite automaton is said to be nonambiguous if for each pair ( p , q ) of states and each finite word w , there exists at most one path going from p to q labeled by w.

As a consequence of Proposition 3, we obtain the following result.

Corollary: Let U be a finite transducer and let S (resp. T ) be the sofic system recognized by the input automaton Q? (resp. the output automaton a). If the automata d is right-closing and the automaton is local, then the sofic systems S and T have the same entropy.

E. Codes

Let A be a finite alphabet. A subset X of the free monoid A* is a code over A if for all n , m 2 1 and x , ; . 'J,, y , , . . ., ym E X , the condition

X , X , - * x , = y l y z * * y,

implies n = m and x , = y , , for i = l , . . . , n .

In other words, a set X is a code if any word in X + can be written uniquely as a product of words in X , that is, if any word in X + has a unique factorization in words in X . For example, the set {a,ab,ba) is not a code and the set {aa, baa, ba) is a code. An important class of codes is the class of prefix codes. A subset X of A* is a prefix set if no element of X is a proper left factor of another element of X . Any prefix set X # ( 1 ) is a code. The free monoid generated by the words of X is denoted by X * .

The following result appears in [5]. Since we shall need it here we reproduce briefly a proof for the sake of completeness.

Proposition 4: Let X be a finite code. Let n j be the number of words of X of length i, 1 I i I N .

1) The two following conditions are equivalent:

a )h(X ' ) > log(k) ,

b) ; > l . i > O

2) If h ( X * ) > log(k) we can find a finite subset Y of X such that h(Y *) = log(k1.

3) If h ( X * ) = log(k) we can construct a maximal prefix code over a k-letter alphabet with the same length distribution as X .

Proofi 1) and 2). The entropy of the language L is log(l/p,) where p L is the convergence radius of the characteristic series f, where

f L ( t ) = card( L n A")t". n t 0

Since X is a code we have [SI

Since X is a finite code px. is the unique real r such that f x ( r ) = 1. And results a) and b) follow from: h ( X * ) > log(k) (resp. = if and only if p x . < l / k (resp. = 1.

3) We suppose that N c n,k- '= 1 .

Let A be a k-letter alphabet. We are going to construct a sequence ( X I ) , ~,~ such that X I c A', card(X,) = n, and the union of the X I is prefix. We first take n , elements of A that constitute X , . Let us consider X,,, constructed for 1 I m I i - 1. Let R = XI U X , U . . . U Xi- I . We find that card(A' - RA*) 2 n,. We choose n, elements in A' - RA* that constitute X I . The set Y = X , U X , U . . . U X , is a prefix code, maximal among the prefix codes of A*, that is, not strictly included in another prefix code of A*. A code of A* is maximal if it is not strictly included in another code of A*. As any prefix code that is maximal among the prefix codes is maximal [SI, Y is a maximal prefix code of A*.

Remark: Proposition 4 is still true when X is only assumed to be a code such that X * is the set of words which are labels of paths of a finite automaton going from a state p to the state p . Indeed it is still true that p x w is the unique real r such that fx(r)= 1 . This fact is a consequence of a lemma by J. Berstel in [9] (p. 94) about the poles of a rational series with nonnegative coeffi- cients.

I > 0

111. THE METHOD OF POLES FOR CHANNEL CODING

We are going to construct a finite transducer for encod- ing digital data from a free k-ary source to a constrained set of available sequences. The set of available sequences iq assumed to be represented by a subshift of finite type. ':he method follows the lines of P. A. Franaszek's work

Let S be a subshift of finite type included in B Z where B is a finite alphabet. The subshift S can be recognized by a finite deterministic local automaton denoted by d. We can assume that the graph of the automaton is strongly connected. Indeed there is always one of the strongly connected components of the automaton that recognizes

[121.

B6AI.: METHOD O F POLES, <'ODINC METIIOD FOR C'ONSTRAINED CHANNE1.S 767

a subshift of finite type included in S with the same entropy. The words of the unconstrained source are se- quences of letters from a finite alphabet A of cardinality k . We denote by L the set of factors or blocks appearing in the bi-infinite words of S.

Let p and q be two integers such that

s h ( L ) > P b ( k ) .

Let L' (resp. B') be the set of words of L (resp. B * ) of length multiple of q and A' be the set of words of A* of length multiple of p. The entropy of L' or S with B' as alphabet is h(L') = qh(L). The entropy of A'* or AZ with A' as alphabet is MA') = p log(k). After changing the alphabets we can now suppose h ( L ) > log(k) where k is the cardinality of the new alphabet A . Encoding and decoding will be realized with a suitable finite transducer labeled in A x B. The associated input automaton that recognizes the subshift A' will be right-closing. The associated output automaton that recognizes a subshift S' c S of B Z will be a local automaton. As mentioned in the remark following Proposition 3, if such a transducer exists we have h(S') = h ( A z ) , and as a consequence h(S) r log (k ) . The rate q / p is called the coding rate. For practical realization, integers p and q and the coding rate must be small, the optimal value of th'k rate q / p being log ( k I/ h( L).

A. Computing a Set of Principal States

The coding rate is now definitively fixed. We are going to compute a set E of states of the automaton d called the set of principal states. The definition of this set and an algorithm to compute it are given by P. A. Franaszek in [12]. The set of principal states is such that for every principal state q one can find an adequate number of paths going from q to another principal state, these paths being of length less than or equal to a fixed number M . We shall successively attempt to find the set of principal states for increasing values of M = 1,2, . . . . For each value of M we shall compute E by successively decreas- ing a set initially equal to the set Q of the states itself.

Let M 2 1 be a fixed integer. We construct a subset E of the set of the states of the automaton d, the set of principal states, such that for every state q of E, one can find a prefix set of paths (that is, no path is the beginning of another one) satisfying the following properties.

1) Every path has the state q as its origin. 2) Every path ends in a state of the subset E. 3) Every path has a length less than or equal to M . 4) If n, is the number of paths of length i , we have

If such a subset E exists, for every state q of E the set of labels of these paths is a prefix code denoted by Y,

with h(Y,*) 2 log(k) (Property 4). A search algorithm can be:

E +- Q , where Q is the set of all states of d test.

The test is the procedure;

Procedure test; Repeat {bool +- false;

FOR each q in E DO

{ I F PE(q) < 1 THEN DO { E +- E - {q } ; bool +-

true)} } UNTIL (bool = false or E =0).

where bool is a boolean variable and PE(q) is the maxi- mum of the sums

n, c , I S I S M

for all choices of prefix sets of paths satisfying conditions 1) to 4).

Remark: We get an intrinsic set at the end of the procedure test. The order of elements of the current set E is in fact without importance.

Two cases appear at the end of the procedure test: the set E is empty or not. We first consider the case E -0. We begin the search again with a new higher value for the constant M . There always exists a value of M such that the set E is not empty when we have h ( L ) > log(k). In fact let q be a state of the automaton d. As d is a deterministic automaton, the set of words that are finite labels of paths going from q to q is X * where X is a prefix code, such that h ( X * ) > log(k). It follows by Proposition 4) that procedure test with M equal to the length of the longest word of a finite code Y c X with h(Y *) = log(k) will give in the worst case E = {q } where q is a state of d.

Let us suppose E #O. Let E = { e , ; . . , e n ) . For each principal state e, and each set of labels of paths satisfying conditions 11-41, one can extract, by Proposition 4, a prefix code X with h ( X * ) = log(k). The length of a code being the sum of the lengths of the words of the code, we denote by Ye, such a code of minimal length associated to the principal state e, . This new optimization condition is denoted by (*I. By Proposition 4 we can also construct a maximal prefix code of A* with the same length distribu- tion. It is denoted by Xe, .

B. The Construction of the Transducer

We are now going to describe the transducer associated to the computation of the set of principal states.

For each principal state e, of E we construct a state of the transducer, still denoted by e, and called a pole. Each word b , . . . b, of Ye, is associated with one word a , - a , of the same length in Xe,, There is a single path 9 of the automaton d labeled by b , - 6, going from e, to an- other principal state e,'. We construct r - 1 new states for the transducer denoted by s , ; . * , s r - , and the edges ( e , , ( a , I b, ), s, ), . . . , (s,, ( a , lb, 1, s, + , ), ( s r - ,, (a,.Ib,), e: 1. We

76X I E R E TRANSACTIONS ON INFORMATION THEORY, vol.. 36, NO. 4, JULY 1990

get a new path in the transducer, Ullhl U , lh, e, -s, - . . * s,- , -e;

The path 9 of the automaton d is called its projection onto d.

We now give two important properties of the trans- ducer denoted by U.

Proposition 5: The output automaton of the transducer U is a local automaton.

Prooj? By Proposition 1, we have to prove that there do not exist two cycles with the same label. We first remark that two cycles with the same label have the same projection onto the automaton d because d is local. Let us suppose that we have two cycles labeled by a word b , . * * b,, represented as follows:

h , h2 h, Po-PI’P~”’ -P ,=Po p & + p ; b + p i . b2 ’ -+p; ht, = p i )

where b,,b,‘ are letters of B and p,,p,‘ are states of the transducer.

First Case: There exists an integer i, 1 I i I n such that p , and p,’ are both pole states of the transducer. The two cycles of the transducer labeled by b,b, + - b,b , . . b, - , in output, one from p, to p , , the second from p,’ to p,‘, have the same projection onto the automaton d. These projections are two cycles of the automaton d. Then p , and p,‘ are the same pole. As a conseqdence we get p k = p i for every integer k because q, is a prefix code.

Second Case: For every integer i, 1 ~ i ~ n , p , and p,’ cannot both be poles. We know by construction of our transducer that there exists at least one integer i (resp. j ) , 1 I i I n (resp. 1 I j I n), such that p , (resp. p ; ) is a pole. We are now only interested with the pairs (r , , s,) where r, is a state of the first cycle and s, a state of the second cycle, exactly one of them being a pole. The two cycles can be represented as follows after renumbering

U 2 U” (ro ,~o)UI‘(r lrs , ) - - * * - ( r n , s , )

where U, E B*, (r,, s,) = (ro, so). Let &= {i/O I i < n, (r, and s,+ I(modn) are poles) or

ts , and r,+I(modn) are pies)}. For i€d with I-, and s,+ I(mod n ) poles, we denote by f(i) the integer j such that s, is the only pole of the ‘‘interval” [s,,sJ+I,s,+2,* *,s,l.

f 4

F i g . 4 Part of two cycles of output of U.

This integer j is still in &. In fact sJ is a pole, sJ+ I is not one, so rJ+ I is a pole. We define likewise f(i) for i E JZ with s, and r,+I(modn) poles. The application f is a permutation of the set &. (cf. Fig. 3 where the poles are circled). Let i be an integer of d? where r, is a pole and s, is not. Let us consider the words U and U defined by Fig. 4, which shows a part of the two cycles of the output of U. For an integer i in A, r, and s, project to the same principal state. We denote by e, this state. We get uu E Ye,(,, and cl E q,; where

Ye: = {w E B + / W E Yf(,)}.

The set Ye: is still a prefix code. The words of Ye: are labels of paths satisfying the Conditions 11-41 for the state e,. In particular 1) and 2) are true because efc , , -u = e, and 4) is true by virtue of the optimization condition ( * 1. In fact, if Condition 4) were false the set Y defined by

y = (&,(,) - (Y,,,,) n uB+ )) U (U}

0 - e : ) < l(Tf(,))

would be a prefix code satisfying conditions 1)-4) and of shorter length than Yet(,,. If f ( X ) is the length of a code X we get

and

c lR?;) < c l ( X f ( , ) )

= c m$ 0 5 I < card(Y) 0 5 I < card( LT)

0 5 I < Cdrd( x )

so there exists at least one integer i E & such that

/(rC:) <Ue,). This contradicts the optimization condition ( * ). 0

Remarks:

1) If the optimization condition ( * ) is not satisfied the result is false as we can see with the following example. Let d be the automaton of Fig. 5. With k = 2 , M = 2 we get E=(1,2} and we can choose Y, = (d ,ab,hg} and Y, = { c , b a , e f } . Then ab is the label of two different cycles of the output of the transducer.

h b

Fig. 3. Permutation f. Fig. 5. Automation .PL.

7 -

BkAI.: METHOI> OF POI.FS, C'OI>IN<i MTTl101> FOK C'ONSTKAINTI> ('ilANN1:l.S 769

2) The optimization condition ( * ) is not used by P. A. Franaszek in [ 151 to guarantee limited error-propa- gation property. His method consists of finding a subset E' of the set of principal states E such that for each state q of E' one can find a set of paths from q to another state of E' satisfying conditions 3) and 4) and the following property: every path that goes through a state of E' terminates there. If such a subset E' exists, error-propagation in decoding will be excluded but, if not, it leads to increase the value of M as we can see with the example next.

Let us consider the subshift of finite type recog- nized by the automaton of Fig. 6. With k = 2, M = 2 we get E=(1,2,3,4,5) and we can choose Y , = Iv, ab, ac, 4, Y2 = Ib, c), Y, = Ih, zt, eg), Y, = { U , U ) , Y5 = {f, w). We are looking for a subset E' satisfying the previous conditions. If 1 P E' then 5 must also be excluded with M = 2. If 5 6G E' then 1, 3, and 4 must be excluded. If 2 E' then 3 is excluded. If 3 P E' then 4 is excluded. If 4 6G E' then 1 is ex- cluded. And if 3,4,5 P E' then 2 is excluded. So there does not exist a subset E' with M = 2.

- - t d Fig. 6.

Proposition 6: The input automaton of the transducer U is right-closing.

Proofi Let I be the maximal length of all codes Xe, associated to the principal states. As these codes are prefix codes, the right-closing condition is satisfied with n = 1. 0

C. How to Encode and Decode

After the construction of the transducer U we choose a start state among the poles. Let q be this state. We suppose that the word to encode is a long enough word of A* denoted by w. There is exactly one prefix U of w contained in the maximal prefix code X , and one path of the input automaton of the transducer labeled by u from q to another pole r . The corresponding output label will be the beginning of the encoded word of w. If w = uw' the construction is iterated from the state r with the word w'. The size of the window necessary to encode the word w is the integer n of the right-closing condition for the input automaton of the transducer.

As the output automaton of the transducer satisfies the local property for an integer n , the decoded word of a

word will be obtained with the transducer and a sliding- block window of length n. As mentioned in Section I , the transducer U is an A" X BZ automation, and the decod- ing is seen as a local map.

As a consequence, every local noise on the encoded word will produce a local error on the decoded word. The coding method is said to be without error propagation.

The method cannot be extended to the limit case of equality of entropies ( h ( S ) = log (card ( A ) ) ) without changing the alphabets and consequently the coding rate. A different method has been found by R. L. Adler, D. Coppersmith, M. Hassner, and B. Marcus (cf. [2] and [17]), which allows the equality case. The method cannot be extended to some sofic systems called almost of finite type sofic systems either (cf. [17] for the definition of almost of finite type sofic systems and another method adapted to them).

IV. EXAMPLES

We first present some examples of problems appearing in magnetic storage where one has to encode information into binary sequences in which the number of 0's occur- ring between each pair of successive 1's has both an upper and a lower bound. The words of A* with A = (0,l) such that there exists at least d and at most k 0's between two 1's are called the [ d , k] words. These constraints are called the [ d , k] constraints.

A. The p, 001 Constraint

We want to encode the words of A N , A = (O,l), into infinite words satisfying the [ 2 , ~ ] constraint. No factor of these words can be 11 or 101. They are recognized by the finite automaton of Fig. 7. The entropy of the associated language or subshift of finite type is less than the entropy of A* but is superior to half of it. We consider the language L of the [2,03] words of even length seen over the new alphabet (00,01,10,11). We get h ( L ) = log(2) and L is recognized by the automaton of Fig. 8 with a = 00, b = 01, and c = 10. With the notations of Section 111 we have Q = {1,2,3) and k = 2. We fix M = 2. If S is a subset f Q and q E Q we denote by P,(q) the maximum

Fig. 7. 12.3~1 constraint.

b

Fig. 8. [2,m] words.

770 I E E E TRANSACTIONS ON INFORMATION THEORY, VOL. 36, NO. 4, JULY 1990

of the sums O b 1 l a I I C

ni - l l i l M c , Ob

Fig. 12. Transducer for [1,3] constraint. for all choices of a prefix set of paths going from q to another state of S (n i is the number of paths of length i). We get

1 1 1 3 '{1,2,3)(') 7 + 7 + 7 = 7 , P(l,2,3)(1) < 1 ,

1 1 1 3 4 4 4 4

P{2,3)( 2) = - + - + - = - < 1 , where state 1 is excluded from the current set of poles,

1 1 1 2 4 4

P(,](3) = - + - + - = 1

We find E = {3) and Y3 = ( a , ba, ca). We can then choose X3 = {0,10,11) and the correspondence (cf. [121)

O a 10 b a . 11 ca

The transducer which gives encoding and decoding is in Fig. 9.

n

O h

Fig. 9. Transducer for [2, m] words.

B. The /1,3] Constraint

The [1,3] words are recognized by the automaton of Fig. 10. The set of [1,31 words of even length, seen over the alphabet of the two letter words ( a = 00, b = 01, c =

10, d = 11) is denoted by L and recognized by the automa- ton of Fig. 11. For a window size (the value of M ) equal to 1, the algorithm gives us a nonempty set of principal states E = {O, 1,2). We choose Yo = ( b , a), Y , = { b , c) , Y2 =

{ b , c}. With the choice of X2 = X, = (0,1} and the associa- tion 0- b and 1 - c as a correspondence between X, and Yl and between X2 and Y,, we can merge state 1 and state 2 of the associated transducer and get the automa- ton given in Fig. 12.

1

Fig. 10. [1,3] constraint.

Fig. 11. [1,3]words.

C. The p, 71 Constraint

The [2,7] words are recognized by the automaton of Fig. 13. The set of [2,7] words of even length, seen over the alphabet of the two letter words { a = 00, b = 01, c =

10, d = 11) is denoted by L and recognized by the automa- ton of Fig. 14. We have h(L) > log(2) and the k = 2. We find a nonempty set of principal states for M = 4 and get E = {2,3). We can choose Y2 = Y3 = Y where Y is the Franaszek code [2]:

ba ca

aba aca cba acba aaca .

1 Fig. 13. [2,7] constraint.

c

Fig. 14. [2,7] words.

The output of the transducer U is in Fig. 15. The trans- ducer can be reduced by indentification of states 2 and 3 after choosing X , = X,. If we construct the transducer

Fig. 15.

77 1 B ~ A L : METHOD OF POLES, CODING METHOD FOR CONSTRAINED CHANNELS

with the following correspondence, we can get a seven states transducer:

11 ba 10 ca 011 aba 010 aca 000 cba 0011 acba 0010 aaca.

D. The Miller Code

We now give an example of coding suitable for some constraints found in some transmission systems. We get a coding known as the Miller coding [19]. The constraint is given by the finite collection of forbidden finite se- quences,

101 010

11111 00000.

We get a subshift of finite type S of words over the alphabet (0,l) recognized by the automaton of Fig. 16. The entropy of the subshift is superior to half of log(2). We change the output alphabet. The new output alphabet is { a = 00, b = 01,c = 10,d = 11). The words of S of even length are recognized by the automaton of Fig. 17. We get eight principal states with M = 1. The enclosed part of the automaton is kept and reduced by identification of

6 6 4d 0 0 0

0 0

Fig. 16. Miller constraint. \

Fig. 17.

Fig. 19. Miller coding.

equal labeled states. We get the automaton of Fig. 18 and finally the transducer of Fig. 19.

V. ANOTHER OPTIMIZATION

To minimize the coding cost, we are looking for trans- ducers as small as possible with coding and decoding windows as small as possible. We are just mentioning without description a method found by D. Perrin to transform a transducer with a right-closing input and a local output, to a transducer with a deterministic input and still a local output. It is related to a result of B. Kitchens I181 according to which an onto map f : S + T , where S and T are subshifts of finite type, is right-closing iff there are two other subshifts of finite type, S' conju- gate to S and T' conjugate to T , where the induced factor map is right-resolving. The new decoding local map is a composition of a short power of the shift with the older decoding local map. This method is described in 161. We get a very small transducer (cf. Fig. 20) for the words [2,7], obtained from the seven states transducer of exam- ple of Section IV-C.

Fig. 20. Transducer for [2,7] constraint.

We finally mention the interesting example of the [O, 11 words. The [0,1] words are recognized by the automaton of Fig. 21. The sequence 00 is forbidden and 00 is a palindromic word. We denote by S the associated sub- shift. Since 3h(S) > 210g(2), we change the input alpha- bet into A = { x = 00, y = 01, z = 10, t = 11) and the output alphabet into B = { a = 000, c = 010, d = 011, f = 101, g =

110, h = 111). Let us consider then the new subshift S of

Fig. 18. Fig. 21. [O, 11 constraint.

772 IEEE TRANSACTIONS ON INFOKMATION THEORY, VOI.. 36, NO. 4, JULY 1990

Fig. 22. We first remark that the subshift S‘ given by the automaton of Fig. 23 is included in S and has an entropy still superior to 2log(2). We next reverse the direction of the arrows. Our algorithm with M = 1 gives us the trans- ducer of Fig. 24. This coding was obtained by A. Gabor [161. e g

f h

Fig. 22. [0,1] words.

ijg$@ h

Fig. 23.

df Wd dg

t/C

Fig. 24. Transducer for [0,1] words.

ACKNOWLEDGMENT

The author would like to thank D. Perrin for pleasant and useful discussions.

REFERENCES

[ l ] R. L. Adler, “The torus and the disk,” IBM J. Res. Dec., vol. 31, no. 2, pp. 149-274, Mar. 1987.

1-21 R. L. Adler, D. Coppersmith, and M. Hassner, “Algorithms for sliding block codes,” IEEE Trans. Inform. Theory, vol. IT-29, pp.

R. L. Adler, J. Friedman, B. Kietchens, and B. Marcus, “State splitting for variable length graphs,” IEEE Trans. Inform. Theory,

R. L. Adler and B. Marcus, “Topological entropy and equivalence of dynamical systems,” Memoir of AMs, vol. 20, no. 219, 1979. M.-P. BCal, “Codes circulaires, automates locaux et entropie,” Theoretical Comput. Sci., vol. 57, pp. 288-302, 1988. -, “Codage, automates locaux et entropie,” Thtse de I’uni- versitt Paris 7, rapport LITP, no. 35, 1988. M.-P. BCal and D. Perrin, “Une caracttrisation des ensembles sofiques,” C.R. Acad. Sc. Pans, vol. 303, pp. 255-257, 1986. J. Berstel and D. Perrin, Theory of Codes. New York: Academic, 1985. J. Berstel and C. Reutenauer, Les shies ratonnelles et leurs lan- gages. Masson, 1984. F. Blanchard and G. Hansel, “Languages and subshifts,” Lecture notes in Computer Science, vol. 192. New York: Springer Verlag,

-, “Systtmes codts,” Theor. Comput. Sci., vol. 44, pp. 17-49, 1986. P. A. Franaszek, “On synchronous variable length coding for discrete noiseless channels,” Inform. Control, vol. 1-J, pp. 155-164, 1969. -, “A general method for channel coding,” IBM J. Res. Dec.,

-, “Construction of bounded delay codes for discrete noise- less channels,” IBM J. Res. Dec., vol. 26, pp. 506-514, 1982. -, “Sequence-state methods for run-length-limited coding,” IBA4 J. Res. Dec., pp 376-383, July 1970. A. Gabor, “Adaptive coding for self-clocking recording,” IEEE Trans. Electron. Comput., Dec. 1967, R. Karabed and B. Marcus, “Sliding-block coding for input- restricted channels,” IEEE Trans. Inform. Theory, vol. 34, no. 1, Jan. 1988. B. Kitchens, “Continuity properties of factor maps in ergodic theory,” Ph.D. thesis, Univ. of North Carolina at Chapel Hill, 1981. C. Macchi and J. F. Guilbert, Tc!lc!informatique. Paris: Dunod, 1983. B. Marcus, “Sofic systems and encoding data,” IEEE Trans. In- form. Theory, vol. IT-31, pp. 366-377, 1985. D. Perrin, “Local maps,” Rapport LITP, no. 3, 1987. A. Restivo, “Codes and local constraints,” Rapport LITP, no. 42, 1987.

5-22, 1983.

vol. IT-32, pp. 108-1 12, 1986.

1985, pp. 138-146.

vol. 24, pp. 638-641, 1980.

[231 C. Shannon, “A mathematical theory of communication,” Bell Syst. Tech. J . , pp. 379-423, 623-656, 1948.

[241 “Small Winchester drives more up to main frame encoding schemes,” Electron. Des., pp. 51-52, Oct 85, 1981.