Parallel communicating grammar systems with terminal transmission

Acta Informatica 37, 511–540 (2001)

c© Springer-Verlag 2001

Parallel communicating grammar systemswith terminal transmission

Henning Fernau

Wilhelm-Schickard-Institut fur Informatik, Universitat Tubingen, Sand 13, 72076Tubingen,Germany (e-mail: [email protected])

Received: 12 April 2000 / 16 October 2000

Abstract. We introduce a new variant of PC grammar systems, called PCgrammar systems with terminal transmission, PCGSTT for short. We showthat right-linear centralized PCGSTT have nice formal language theoreticproperties: they are closed under gsm mappings (in particular, under inter-section with regular sets and under homomorphisms) and union; a slightvariant is, in addition, closed under concatenation and star; their power liesbetween that ofn-parallel grammars introduced by Wood and that of ma-trix languages of indexn, and their relation to equal matrix grammars ofdegreen is discussed. We show that membership for these language classesis complete for NL. In a second part of the paper, we discuss questionsconcerning grammatical inference of these systems. More precisely, weshow that PCGSTT whose component grammars are terminal distinguish-able right-linear, a notion introduced byRadhakrishnan andNagaraja in [33,34], are identifiable in the limit if certain data communication informationis supplied in addition.

Motivation, definitions and examples

1 Introduction

Parallel communicating grammar systems (PCGS, for short) were intro-duced in [31] in order to investigate concepts like parallelism, synchroniza-tion and data communication with formal language theoretic means.

Unfortunately, the language families introduced in this fashion are ratherintricate from a formal language point of view, even if one restricts oneselfto right-linear grammar components, as has been done, for instance, it the

512 H. Fernau

alreadyquoted introductoryPCGSpaperofPaunandSantean.Only recently,several closure properties have been shown by Autebert [4] by means ofspecial techniques. There are few non-trivial known inclusion propertieswith respect to other language families studied in the literature. As regardsto complexity questions, in particular the fixed membership problem, theinclusion of so-called centralized regular PCGSwithin the complexity classNL (defined by languages acceptable by nondeterministic Turing machineswhose work tape is only of logarithmic length in terms of the length of theinput word) has been shown by Cai in [10] by using a special construction.On the other hand, the technique used by Abrahamson, Cai and Gordon toshow NL hardness for so-called coherent PCGS heavily makes use of non-centralized features and is, hence, not applicable to our case [1]. In fact,NL hardness of the fixed membership problem for non-centralized regularPCGS with two components already follows by its inclusion of the linearlanguages proved in [12] in combination with the well-known NL hardnessresult for linear languages due to Sudborough [42].

We will discuss a variant of PCGS which we call PCGS with terminaltransmission, PCGSTT for short, a model where only transmissions of ter-minal strings are allowed in order to exclude the influence of the queriedcomponent grammar on the querying component grammar. Systems withthis property were already investigated by Paun in [30] under the nameter-minally synchronizedPCGS. They appear to have rather weak descriptivecapacity. In our version, we enhance the power of themechanismby the sim-ple trick that we consider the so-called query symbols formally as terminalsymbols (and not as nonterminal symbols).We further show that right-linearPCGSwith terminal transmission (in our definition) have rather nice formallanguage properties, including simple hierarchical relations to well-knownregulated formal language classes and complexity classes, contrary to thecase of the classical variant of regular PCGS. We will treat these questionsin the first part of this paper.

Moreover, there is also another motivation for discussing this variantwhich is intrinsic to PCGS: Parallel synchronized computation should befree from side-effects, since this allows the calling processor to continueits work without waiting for the end of the transmission of the results fromthe called processor. Otherwise, it is not reasonable to assume that com-munication (such as derivation) takes only one step. We can also think ofPCGSTT as modelling data independence features by way of grammar sys-tems, a notion well-studied in the parallel complexity community under thename ofowner read, owner write PRAM(see [25,26,38]). Data indepen-dence has been studied within the framework of finite automata theory andrelation towards complexity theory in [22,32]. Observe that data indepen-dence is a particularly useful and reasonable assumption when considering

Parallel communicating grammar systems with terminal transmission 513

both derivation steps and communication steps of a grammar system to hap-pen in unit time steps since, then, a possibly long communicated terminalstring can be safely “buffered” somewhere, and the derivation of the systemmay proceed, since the communicated string has no influence on the furtherdevelopment of the system.

Another aim and motivation of the present study was to investigate theinferrability of certain right-linear PCGS language families, given positivesamples only, within the learning model “identification in the limit” pro-posed by Gold [20]. This is an important issue from two different angles:(1) People who like to apply Gold’s learningmodel should be given the pos-sibility to choose among a great variety of grammar or automata formalismsthe one which meets their needs best. Concerning PCGS language families,applications where parallelism, synchronization and data communicationare involved are good candidates. (2) It is always a challenge for theoreti-cians to make their work as applicable as possible. For reasons explainedbelow, we did not foresee good opportunities to develop a learning theoryfor the PCGS formalism as defined classically. Therefore, we introducedthe present model. In particular, the data independence feature seems to beneeded for inference purposes. We will come to this issue in the secondpart of this paper. We conclude that part with a detailed possible applicationscenario originating from discussions with people working in the hardwaremanufacturing industry.

A preliminary version of this paper is included in the proceedings of theconference Grammar Systems 2000, see [15].

2 Definition of PCGS with terminal transmission

Definition 1 A right-linear parallel communicating grammar system withterminal transmissionwith n components, wheren ≥ 1 (a PCGSTT forshort), is an(n + 3)-tuple Γ = (N,K,Σ,G1, . . . , Gn), whereN is anonterminal alphabet,Σ is a terminal alphabet andK = {q1, q2, . . . , qn}is an alphabet of query symbols.N , Σ andK are pairwise disjoint sets,Gi = (Ni, Σ∪K,Pi, Si), 1 ≤ i ≤ n, called the components ofΓ , are usualChomsky grammars with nonterminal alphabetNi ⊆ N , terminal alphabetΣ ∪K, a set of productionsPi and axiom (or start symbol)Si. The rulesare of the formA→ wB, wherew ∈ Σ∗K∗ andB ∈ {λ} ∪N . We requirethatN = N1 ∪ . . . ∪Nn.G1 is said to be the master grammar (or master)of Γ.

Observe that query symbols are formally considered as additional ter-minal symbols. Of course, it would also be possible to admit context-freeor linear grammar components, but since we are going to restrict our con-

514 H. Fernau

siderations to the regular case in the following, we only defined right-linearPCGSTT above.

Definition 2 Let Γ = (N,K,Σ,G1, . . . , Gn), n ≥ 1, be a PCGSTT. Ann-tuple(x1, . . . , xn), wherexi ∈ (Σ ∪ Ni ∪K)∗, 1 ≤ i ≤ n, is called aconfigurationof Γ . (S1, . . . , Sn) is said to be theinitial configuration.

PCGSTT change their configurations by performing direct derivation steps.

Definition 3 Let Γ = (N,K,Σ,G1, . . . , Gn), n ≥ 1, be a PCGSTTand let(x1, . . . , xn) and(y1, . . . , yn) be two configurations ofΓ. We saythat(x1, . . . , xn) directly derives(y1, . . . , yn), denoted by(x1, . . . , xn) ⇒(y1, . . . , yn), if one of the following two cases hold:

1. Rewriting step: There is noxi which contains any query symbol, that is,xi ∈ (Ni ∪Σ)∗ for 1 ≤ i ≤ n.In this case,yi is obtained fromxi by a direct derivation step inGi, thatis,xi ⇒Gi yi.Forxi ∈ Σ∗, we havexi = yi in the normal mode. In thefully synchro-nized mode, we block the derivation of the grammar system ifxi ∈ Σ∗.1We also write(x1, . . . , xn) ⇒rew (y1, . . . , yn) in the case of a rewritingstep.

2. Communication step: There is somexi, 1 ≤ i ≤ n, which contains atleast one occurrence of query symbols.Letxi be of the formxi = zqi1qi2 . . . qitA, wherez ∈ Σ∗,A ∈ N ∪{λ}andqil ∈ K, 1 ≤ l ≤ t. In this case,yi = z1xi1xi2 . . . xitA if, for all1 ≤ l ≤ t, xil ∈ (Σ ∪ K)∗. If a would-be communicated stringxilcontains a nonterminal symbol, the derivation blocks. Furthermore, allnon-querying but queried componentsj are reset, i.e.,yj = Sj in thesecases.In thereturning mode, all non-querying components are reset, i.e.,yi =Si in these cases.In the non-returning mode, all non-querying non-queried componentscontinue their work, i.e.,yi = xi in these cases.We also write(x1, . . . , xn) ⇒com (y1, . . . , yn) in the case of a rewritingstep.

Let⇒:=⇒rew ∪ ⇒com.

The first case is the description of a rewriting step: if no query symbol ispresent in any of the sentential forms, then each grammar component usesone of its rewriting rules, except for those which have already produced a

1 In other words, a terminal string, once generated, will block the derivation in fullysynchronized mode if it is not containing a query symbol or if it is not queried in the nextstep.


terminal string. If a terminal string has been produced, we distinguish twocases. Observe that the derivation is blocked if a sentential form of somecomponent grammar is not a terminal string, but no rule can be applied toit.

The second case describes a communication step: if some query symbol,sayqi, appears in a sentential form, then rewriting stops and a communica-tion step must be performed. The symbolqi must be replaced by the currentterminal sentential form of componentGi, sayxi. Observe that in classicalPCGS, query symbols are not communicated. Instead, first only compo-nents querying other components whose sentential form does not containany query symbol may get their queries satisfied, and the other componentswhose sentential forms contain query symbols stay with these same senten-tial forms until they, in the next steps, eventually may satisfy their queries. Itis seen quite easily that our normal model is indeed equivalent to this usualdefinition in the case of non-returning (classical) systems.

To finish a communication step, the components are reset according tothe choice between returning or non-returning mode. Note that this is aslight difference to the “classical” model of returning PCGS, where onlyqueried components resume their work on the axiom, but when consideringterminal transmission, it seems to be unreasonable to assume that queriedcomponents maintain their (terminal) sentential forms; at least, they shouldrestart.

If a circular query appears, the derivation continues forever without pro-ducing a terminal string as a result.

Let⇒rew and⇒com denote a rewriting step and a communication step,respectively. Let⇒∗ denote the reflexive transitive closure of⇒.

Definition 4 Let Γ = (N,K,Σ,G1, ..., Gn) be a PCGSTT with mastergrammarG1 and let(S1, ..., Sn) denote the initial configuration ofΓ . Thelanguagegenerated by the PCGSTTΓ is

L(Γ ) = {α1 ∈ Σ∗ | (S1, . . . , Sn) ⇒∗ (α1, . . . , αn)}.

Thus, the generated language consists of the terminal strings appearing assentential forms of the master grammarG1.

Our studies in the present paper will focus on returning PCGSTT whicharecentralized, i.e., only themaster grammarmay introduce query symbols.Hence, circular queries will never occur. Furthermore, the communicationin these parallel communicating grammar systems can be called truly data-independent, since both the (indirect) influences on the calling grammar viacommunicated query symbols and via nonterminals is explicitly excluded.

Observe that, in the case of centralized systems, non-returning fullysynchronized PCGSTT have a simple communication structure: In each

516 H. Fernau

derivation of some terminal string, there is at most one communication stepinvolved, and this would have been the last derivation step in that derivation.

Let the class of returning centralized right-linear PCGSTT with at mostn right-linear components, be denoted byPCnGSTT. Since no confusionwill arise, the class of languages generated by these systems will be denotedby PCnGSTT, as well. When an arbitrary number of components is con-sidered, we use∗ in the subscript instead ofn. We add fs in our notationif we consider full synchronization, so that, for example, we arrive at thelanguage classPC∗GSTTfs.

3 Examples of PCGSTT

Example 5ConsideraPCGSTTwith3components,where themastergram-mar contains the rulesS1 → aS1 and S1 → aq2q3, G2 has the rulesS2 → bS2 andS2 → b andG3 has the rulesS3 → cS3 andS3 → c.This PCGSTT generates

{ akb�cm | 1 ≤ �,m ≤ k }in the normal mode and

{ ambmcm | 1 ≤ m }when viewed as fully synchronized.

Example 6ConsideraPCGSTTwith2components,where themastergram-mar contains the rulesS1 → S1 andS1 → q2q2 andG2 has the rulesS2 → aS2, S2 → bS2 andS2 → λ. This PCGSTT generates

{ww | w ∈ {a, b}∗},both in normal and fully synchronized mode.

These examples show that particularly the full synchronizationmode canbe of help to define languages which are “complicated” when consideringits Chomsky type.

Example 7Consider the following example of a PCGSTT with three com-ponents:Γ = ({S,A}, {q2, q3}, {a, $}, G1, G2, G3), whereGi has rule setPi with:

P1 = {S → S, S → q3A,A→ A,A→ $q2q3q2A,A→ q3},P2 = {S → S, S → $A,A→ aA,A→ a} and

P3 = {S → S, S → $S, S → aS, S → $, S → a}.


Due to the looping rulesS → S, the fully synchronized mode is equiv-alent, in this case, to the normal mode. Now, consider the languageL =L(Γ ) ∩ ($a+$$a+)+$. Let w ∈ L. w encodes an instance of the graphaccessibility problem for directed graphs in the following way:

– The nodes of the graph are coded as subwords$ai$, designating nodenumberi in this way.

– Each subword of the form$ai$$aj codes an arc from nodei to nodej.– The graph accessibility problem asks whether there is a directed path

from some specified start nodei0 to some specified target nodeit. Inour coding, we simply assume that the first coded nodei0, which isrepresented as the prefix of the form$ai0$ in w, is the start node, andthat the last coded nodeit, which is represented as the suffix of the form$ait$ in w, is the target node.

Now, if w ∈ L, thenw encodes an instance of the graph accessibilityproblem; in particular, this means that there exists at least one directedpath from the encoded start node to the target node of the graph. This isguaranteed by the two calls of the second grammar component. The thirdcomponent is only necessary to “hide away” the created path within otheredges generated by the third component.

On the other hand, any instance of the graph accessibility problem canbe coded in this way.

This third example shows that our systems can generate languages ofa higher computational complexity than any single component may create,since regular languages are complete for the complexity class NC1, whilethe graph accessibility problem is the paradigmatic hard problem for NL,nondeterministic logarithmic space.

Lemma 8 For all n ≥ 1, PCnGSTT ⊆ PCnGSTTfs.

Proof. In each non-master component, one has simply to introduce a “loop-ing rule” of the formS → S, whereS is the start symbol of that component.In this way, the component may defer its termination to the required syn-chronization point. �

Part 1: Formal language issues

4 Closure properties

Closure properties are known for regular PCGS only by recent results, see[4]. Their proofs generally require the combination of several techniquesspecific to PCGS. We give here positive closure results for PCGSTT, basi-cally using only standard arguments.

518 H. Fernau

First, recall the notion of a generalized sequential machine (gsm): a gsmγ = (Q,Σ,∆, δ, q0, Qf ) has state alphabetQ, input alphabetΣ, outputalphabet∆, start stateq0 ∈ Q, a set of final statesQf ⊆ Q and a finitetransition relationδ ⊆ Q × Σ ×∆∗ × Q, whose rules are also written asqa → wq′, whereq, q′ ∈ Q, a ∈ Σ andw ∈ ∆∗. These rules define arewriting system: a string of the formuqav with u ∈ ∆∗, q ∈ Q, a ∈ Σ andv ∈ Σ∗ yieldsuwq′v, writtenuqav ⇒γ uwq

′v, when the ruleqa → wq′is applied. Then, forL ⊆ Σ∗, let γ(L) = {w ∈ ∆∗ | ∃v ∈ Σ∗∃qf ∈ Qf

q0v∗⇒γ wqf }, where ∗⇒γ is the reflexive transitive closure of⇒γ .

Theorem 9 PC∗GSTT andPC∗GSTTfs are closed under gsmmappings.

Proof. (Sketch) A variant of the “pair construction” (remembering the orig-inal plus the current state of the simulated gsm) may be used for the proof.The non-master components are multiplied (times the square of the num-ber of states of the considered gsm), so that the appropriate gsm simulatingcomponent can be queried by the master.

More precisely, letΓ = (N,K,Σ,G1, . . . , Gn) be a PCGSTT(fs).Without loss of generality, we can assume that all non-master compo-

nents generate infinite languages since, otherwise, calls to components gen-eratingfinite languagescanbe incorporateddirectly in themaster componentand, furthermore, blockages for the grammar derivation due to componentsgenerating finite languages can be simulated within the master componentby simple counting.

Furthermore, we can assume that the simulated gsmγ is “completelydefined” in the sense that, for each stateq and for each input symbola, thereare a wordw over the output alphabet and a stateq′ such thatqa⇒γ wq

′.For simplicity, we may assumeK = {2, . . . , n}, i.e., i ∈ K can be

considered as label ofGi. Consider a gsmγ = (Q,Σ,∆, δ, q0, Qf ).Wedefine a PCGSTT(fs)

Γ ′ = (N ′,K ′ = K ×Q×Q,∆,G′1, . . . , G

′n′),

with n′ = 1 + (n− 1)|Q|2.The new master grammarG′

1 contains, for each rulep of G1, severalrules according to the following two cases.Case 1:p = A → wi1 . . . ikB, w ∈ Σ∗, ij ∈ K, B ∈ N . Then,G′

1contains all rules of the form

(A, q) → w′(i1, q1, q2) . . . (ik, qk, qk+1)(B, qk+1),

whereq, q1, . . . , qk+1 ∈ Q andqw∗⇒γ w

′q1.Case 2:p = A → wi1 . . . ik, w ∈ Σ∗, ij ∈ K. Then,G′

1 contains all rulesof the form

(A, q) → w′(i1, q1, q2) . . . (ik, qk, qk+1),


whereq, q1, . . . , qk+1 ∈ Q, qk+1 ∈ Qf andqw∗⇒γ w

′q1.If S is the start symbol ofG1, then(S, q0) is the start symbol ofG′

1.Consider a non-master grammarG′

j with label (i.e., corresponding querysymbol (i, q, q′)). Then,G′

j simulatesGi and, in parallel, it simulatesγstarting from stateq and terminating in stateq′. If A → wB, w ∈ Σ∗,B ∈ N is a rule inGi, then(A, q) → w′(B, q′) is a rule inG′

j , where

q, q′ ∈ Q andqw∗⇒γ w

′q′. If A → w with w ∈ Σ∗ is a rule inGi, then(A, q) → w′ is a rule inG′

j , whereq ∈ Q andqw∗⇒γ w

′q′. If S is the startsymbol ofGi, then(S, q) is the start symbol ofG′

j . �Werefrain fromgivinga formal inductiveproofargumentof theconstruc-

tion in the previous proof. Instead, we mention the following observations:

1. The returning mode is essential for the correctness of the constructionsince, otherwise, the synchronized simulation of one grammar compo-nentGi of the original grammar by a number of grammar componentsG′

j of the simulating grammar couldn’t be guaranteed.2. The simulation works both in normal and in fully synchronized mode.3. Theconstructiondoesnot transfer to themoregeneral caseofarbitrary ra-

tional transductions, since desynchronization effects due toλ-transitionmight occur. Only subsequential transductions where, basically, a regu-lar set may be appended at the end of a string translation by a gsm, can besimulated; this result immediately follows from the gsm-closure shownabove and the catenation closure proved below. For different notions oftransductions, we refer to [5].

By making use of standard constructions, one can show:

Theorem 10 For eachn ≥ 1, PCnGSTT andPCnGSTTfs are closedunder homomorphisms. �Theorem 11 PC∗GSTT andPC∗GSTTfs are closed under union.

Sketch of Proof.Assume thatL1 is generated by aPCGSTTG1 withn1 com-ponents with axiomsA1, . . . , An1 , and thatL2 is generated by a PCGSTTG2 with n2 components with axiomsB1, . . . , Bn2 . Wemay assume that thenonterminal alphabets ofG1 andG2 are disjoint. The PCGSTTG generat-ing the unionL = L1 ∪ L2, hasn = n1 + n2 − 1 components. Assuminga new axiomS1 for the master component, the master grammar has allrules of the master grammars ofG1 andG2 (where the query symbols haveto be adapted in an obvious manner) plus the new start rules (which playthe role of nondeterministic choice rules in the classical textbook construc-tions)S1 → w, for all rulesA1 → w of the master component ofG1 andfor all rulesB1 → w of the master component ofG2. The second throughn1th components are the non-master components ofG1, and the(n1 + 1)th

520 H. Fernau

through(n1 +n2 −1)th components are the non-master components ofG2.Querying rules ofG1 andG2 (which are now collected in the new mastercomponent ofG) have to be updated accordingly. �Theorem 12 PC∗GSTT andPC∗GSTTfs are closed under plus and staroperations.

Sketch of Proof.We only show the construction for the plus operation. Dueto the closure under union, the claim concerning the star operation followsimmediately.

Let a PCGS generatingL consist ofn components. We describe a PCGSforL+ withn+1 components in the following. The(n+1)th component hasjust the rulesSn+1 → Sn+1 andSn+1 → λ and is used as a “synchronizer”.For every ruleA → w wherew = sq with s ∈ Σ∗ and q ∈ K∗, putA → sqqn+1S1 as an additional rule into the master grammar. In this way,all components (besides the master component) reset their work and startnew derivations from their start symbols again. �

Observe that the “blow-up” of the number of components is only due tothe fact that we have to synchronize all components by querying a “dummycomponent”. This is not necessary if all terminal rulesA → w of the orig-inal master contain query symbols. This can be seen best by the followingexample:

Example 13{anbn | n > 0}∗ ∈ PC2GSTTfs.

Proof. ConsiderΓ = (N,K,Σ,G1, G2), whereG1 has the rulesS1 → λ,S1 → aA, S1 → aq2S1, A → aA andA → aq2S1, andG2 has the rulesS2 → bS2 andS2 → b. �

Again, observe that the returning mode feature is essential for the cor-rectness of the construction in Theorem 12; otherwise, the “synchronizer”would not work.

Theorem 14 PC∗GSTT andPC∗GSTTfs are closed under concatena-tion.

Sketch of Proof.It is clear that the language families in question are closedunder concatenation with finite languages; this can be easily done withinthe master component alone.

Hence, we can assume that both languages which are to be concatenatedare infinite. This allows us to combine the proofs given for the closure underunion and under star: We start simulating both involved grammar systems inparallel (as in the union construction), but with the master component firstsimulating the first PCGS master component and then switching (as in thestar construction) to the simulation of the second PCGSmaster component.


In this switching phase, all non-master components are reset by querying adummy component generating the empty string as in the star construction.We need the assumption that the original languages are infinite in order toprevent unwanted derivation blockages in the full synchronization mode.

�Since the synchronization feature of PCGS poses some problems, it is

not known whether the corresponding language classes are closed underinverse homomorphism. In particular, it is open whetherPC∗GSTTfs orPC∗GSTT forms a full AFL (abstract family of languages). Furthermore,closure under intersection and complementation is not fully explored.

5 Hierarchy relations

We can prove thatPC∗GSTTfs strictly include the parallel right-lineargrammars defined by Wood and Rosebrugh [35–37,48] and are strictly in-cluded in the class describable by context-free matrix grammars of finiteindex. This immediately yields that the fixedmembership problem for right-linear PCGSTT is in NL, a result which has also been proven in the classicalcase by Cai, see [10] and, more generally, [9]. We elaborate on this in thefollowing section. Moreover, we strongly conjecture thatPC∗GSTTfs areincomparable with right-linear simple matrix languages, an issue discussedmore thoroughly in the first part of the conclusions.

Definition 15 (n-parallel right-linear grammar) Forn > 0, ann-parallelright-linear grammar(n-PRLG) is an(n+3)-tupleG = (N1, . . . , Nn, Σ, S,P ), whereNi, 1 ≤ i ≤ n, are pairwise mutually disjoint nonterminalalphabets,Σ is the terminal alphabet,S /∈ N ∪Σ is the start symbol whereN = N1 ∪ . . . ∪Nn, and

P ⊆ N × (Σ∗N ∪Σ+) ∪ {S} × (N1 . . . Nn ∪Σ∗)

is a finite set of rules, writtenX → x.The yield relation is defined as follows: forx, y ∈ (N ∪ Σ ∪ {S})∗,

x ⇒ y iff either x = S andS → y ∈ P , or x = y1X1 . . . ynXn, y =y1x1 . . . ynxn, whereyi ∈ Σ∗, xi ∈ Σ∗Ni ∪Σ+,Xi ∈ Ni andXi → xi ∈P , 1 ≤ i ≤ n. We extend⇒ to

∗⇒ in the usual way.L ⊆ Σ∗ is ann-parallelright-linear language(n-PRLL) iff there exists ann-PRLGG generatingL.The family ofn-PRLL is denoted byRn. LetR∗ =

⋃n≥1 Rn.

Rosebrugh and Wood have shown thatRn is strictly contained in thefamily of n-right-linear simple matrix languages, a class which we do notintroduce formally here. We refer the reader to [11,23,41].2 Below, instead,

2 The formalism of right-linear tuple languages is equivalent to right-linear simple matrixlanguages, see [27,37]. The language class has been named “equal matrix languages” bySiromoney [41].

522 H. Fernau

we introduce matrix languages of indexn which, in turn, generalize then-right-linear simple matrix languages.

Loosely speaking, one can think of ann-PRLG as consisting ofn right-lineargrammarsworking inparallel. There isonesubtletyonehas toobserve:at the very beginning, there is some sort of coordination between the gram-mars, since the start points of the grammars are selected in a coordinatedmanner. If this feature is excluded, one arrives at the so-calledn-parallelfinite state languages which form the proper subclassJn of the classRn,see [47,48].

We would like to discuss briefly the relations to terminally synchronizedcentralized PCGS as introduced by [30, Section 5] shortly, supplementing[30] in this way. Without giving definitions and details here, wemention thefollowing properties:

1. For every terminally synchronized centralized PCGS with right-linearcomponents, there exists an equivalent system of the same kind withonly two components. Namely, since there will be only (at most) onequery of some non-master component at the end of the derivation, onenon-master component can simulate a bunch of non-master componentsby nondeterministic choice at the very beginning of its computation. Thisobservation somehow sharpens [30, Theorem 6] for our purpose.

2. The language class derivable by terminally synchronized centralizedPCGS with right-linear components are strictly included in the classPC2GSTT. Due to the first property, we may assume that the termi-nally synchronized centralized PCGS which we have to simulate hasonly two components. Now, interpreting the query symbol as a terminalsymbol proves the inclusion. The strictness may be seen through Exam-ple 6, since the systems according to Paun’s definition can only generatecontext-free languages, see [30, Corollary 1].

3. The language class derivable by terminally synchronized centralizedPCGS with right-linear components are included in the classR2. Actu-ally, an argument as in the previous point applies.

Theorem 16 R∗ ⊂ PC∗GSTTfs.

Proof. LetG = (N1, . . . , Nn, Σ, S, P ) be ann-PRLG. LetN =⋃n

i=1Ni.We give a simulating PCGSTTfs with|N |+1 components, namely amastercomponentM and componentsGA, whereA ∈ N . LetqA denote the querysymbol for componentGA.M has the following rules:

– a “waiting rule”S → S;– for every ruleS → y with y ∈ Σ∗, S → y is contained in the master’s

rule set; and– for every ruleS → X1 . . . Xn with Xi ∈ Ni, S → qX1 . . . qXn is in themaster’s rule set.


ComponentGA, whereA ∈ Ni, has the rule setP ∩Ni × (Σ∗Ni ∪Σ+).The strictness is easily seen by making use of Example 13. The fact that

that language is not contained inR∗ can be seen by noticing that that lan-guage is not evena linear simplematrix language, see [11, Lemma1.5.6(iii)].

�It is moreover possible to characterizeR∗ languages in terms of

PC∗GSTTfs:

Theorem 17 L ∈ R∗ iff there exists a PCGSTTfsΓ withL(Γ ) = L whichsatisfies the property that the query alphabetK ofΓ can be partitioned inton disjoint subsetsK1, . . . ,Kn such thatΓ ’s master contains only one formof rules besides the waiting ruleS → S, namely, rules of the formS → k,whereS is the master’s start symbol andk ∈ K1 . . .Kn is a string of querysymbols.

Sketch of Proof.The previous theorem shows that eachR∗ language can besimulated by a PCGSTTfs of the required form.

On the other hand, if we are given a PCGSTTfs of the required form, letus assume that all nonterminal alphabets of all components are pairwise dis-joint. We then put all rules of all non-master components into the simulatingPRLG rule set plus rules of the formS → X1 . . . Xn iff S → qi1 . . . qin isa query rule in the master component of the simulated PCGSTTfs andXj

is the start symbol of componentij for 1 ≤ j ≤ n. �Furthermore, we mention without proof:

Theorem 18 L ∈ Jn iff there exists aΓ ∈ PCnGSTTfs, withL(Γ ) = Lsuch thatΓ ’s master contains only one form of query rules, namely, rulesof the formA → γq2 . . . qn, whereA is some nonterminal symbol of themaster andγ is some terminal string. �Definition 19 (matrix language of indexn) Amatrix grammar(cf. [11])is a quintupleG = (N,Σ,M,S), whereN , Σ, andS are defined as inChomsky grammars (the alphabet of nonterminals, the terminal alphabetand the axiom) andM is a finite set ofmatrices, each of which is a finitesequence

m : (α1 → β1, α2 → β2, . . . , αr → βr), r ≥ 1,

of context-free rewriting rules overV = N ∪ Σ. For some wordsx andy in V ∗ and a matrixm : (α1 → β1, α2 → β2, . . . , αr → βr) inM , wewrite x =⇒

my (or simplyx ⇒ y if there is no danger of confusion) iff there

are stringsx0, x1, . . . , xr (called intermediate sentential forms) such thatx0 = x, xr = y, and for1 ≤ i ≤ r,

xi−1 = zi−1αiz′i−1, xi = zi−1βiz

′i−1 for somezi−1, z

′i−1 ∈ V ∗.

524 H. Fernau

The language generated byG is defined asL(G) = {w ∈ Σ∗ | S ∗⇒ w }.G is of index n (cf. [11,28]) if every terminal word derivable inG

has a derivation where each of its intermediate sentential forms has lessthan or equal ton occurrences of nonterminal symbols. The correspondinglanguage class is denoted byMAT n(CF) or MAT ∗(CF) if the index isnot important.L ∈ MAT ∗(CF) is also called a matrix language offiniteindex.

Theorem 20 PC∗GSTTfs ⊆ MAT ∗(CF).

Proof. We sketch the simulating matrix grammar in the following. LetΓ = (N,K,Σ,G1, . . . , Gn) be aPCnGSTTfs. We may assume that allnonterminal alphabets of the components of the simulated PCGSTTΓ arepairwise disjoint. Moreover, without loss of generality we can assume thatall non-master components when considered “stand-alone” (i.e., as usualright-linear grammars) generate infinite languages. LetSi be the axiom oftheith componentGi. Now, we can describe the simulating matrices:

Start rules: If there is a ruleA → sqi1 . . . qikB in the master grammar ofthe simulated PCGSTT whereA ∈ N andB ∈ N ∪ {λ}, s ∈ Σ∗ andqij ∈ K, then we put a start matrix of the form

(S → (S1, i1 . . . ik, s)sSi1 . . . SikE)

into the matrix set of the simulating grammar, whereE = λ if B = λandE is a new distinguished nonterminal, otherwise. In this way, anondeterministic guess of the querying rule the master will choose sometime in the future is made. This starts the simulation of the grammarcomponents which are going to be queried in the correct way. Note thatwe do not have to worry about simulating grammar components whichare not going to be called, since they will restart after the future query(of other components) due to the returning mode.If there is a ruleA→ s in themaster grammar of the simulated PCGSTTwhereA ∈ N and s ∈ Σ∗, then we put a start matrix of the form(S → S1), as well as a terminating matrix of the form(A→ s), into thesimulating matrix grammar.

Simulation of a normal rewriting step: Assume rulesA1 → α1B1 fromG1,r(j) = Aij → αijBij fromGij ,A1, Ai1 , . . . , Aik , B1, Bi1 , . . . , Bik∈ N , where identical rulesr(j) = r(j′) are selected ifij = ij′ . Suchrules are simulated in the matrix grammar by the matrix

((A1, i1 . . . ik, s) → α1(B1, i1 . . . ik, s),Ai1 → αi1Bi1 , . . . , Aik → αikBik),

where the indicesi1, . . . ik and the terminal strings come from the firstcase discussed in the start rules;


If there is a ruleA→ s in themaster grammar of the simulatedPCGSTT,we add the matrix(A1 → α1B1) to the matrix set.

Simulation of a communication stepof the formA → sqi1 . . . qikB inthe master grammar of the simulated PCGSTT, whereA ∈ N andB ∈N ∪ {λ}, s ∈ Σ∗ andqij ∈ K: Let r(j) = Aij → αij be terminatingrules ofGij , i.e., αij ∈ Σ∗, where identical rulesr(j) = r(j′) areselected ifij = ij′ .In the caseB �= λ, consider the matrix

((A, i1 . . . ik, s) → λ,

Ai1 → αi1 , . . . , Aik → αik ,

E → (B, j1 . . . j�, t)tSj1 . . . Sj�E′),

where we assume that there is a ruleB → tqj1 . . . qj�C in the master

grammar of the simulated PCGSTT, whereB ∈ N andC ∈ N ∪ {λ},t ∈ Σ∗ andqji ∈ K, andE′ = λ if B = λ with E′ = E otherwise.In the caseB = λ, consider the matrix

((A, i1 . . . ik, s) → λ,Ai1 → αi1 , . . . , Aik → αik).

Observe that the simulating matrix grammar has finite index, but this indexneednot correspond to thenumber of components of the simulatedPCGSTT,as the following example shows. �Example 21The language sequenceLn = { (amb)n | m > 0 } has an ar-bitrarily large matrix grammar index (refer to the proof of Theorem 3.1.7in [11]), but each of these languages lies inPC2GSTTfs (more specifi-cally inPC2GSTT), since the master needs only to query the same secondcomponentn times.

In view of the results of the preceding section and keeping in mind thefact thatMAT ∗(CF) forms an AFL, it would be interesting to see a proofof the conjectured strict inclusion ofPC∗GSTTfs within MAT ∗(CF), aswell as of the strictness of the inclusion of the smallest AFL containingPC∗GSTTfs within MAT ∗(CF).

Due to [11, Lemma 3.1.5], we may conclude:

Corollary 22 Each language inPC∗GSTTfs is semilinear.

6 Remarks on complexity

We now turn to the complexity of the membership problem. Actually, wedeal with the so-called fixed membership and the non-emptiness problems.The first of these problems fits profits from the language-theoretic hierarchyconsiderations of the preceding section. More precisely, we consider thefollowing membership problems:

526 H. Fernau

Fixed membership with fixed number of componentsin the case ofPCGS:Let a PCGSG with n components be fixed.For given wordw, isw ∈ L(G)?

Fixed membership with fixed index in the case of matrix grammars offinite index:Let a matrix grammarG of indexn be fixed.For given wordw, isw ∈ L(G)?NL denotes the class of languages which can be accepted by nonde-

terministic Turing machines whose working tape is logarithmically spacebounded.

It is known that the fixed membership problem with fixed index is NLcomplete in the case of programmed grammars of finite index, see [16,Fig. 1]. This implies, in particular, thatMAT ∗(CF) is contained in NLdue to [11,39]. Fortunately, the classical techniques developed by Sudbor-ough are applicable [43,42] to our language classes, so that we can showNL-completeness for our variant rather straightforwardly. Namely, we haveshown in Example 7 how to encode the graph accessibility problem of di-rected graphs into a PCGSTT with only three components. Hence, we canconclude:

Corollary 23 For eachn ≥ 3,PCnGSTT andPCnGSTTfs are completefor NL. �

SincePC1GSTT = PC1GSTTfs coincides with the regular languages,we may state:

Corollary 24 PC1GSTT andPC1GSTTfs are complete forNC1. �(The complexity class NC1 is defined via special forms of circuits. For

further information, the reader is referred to any modern textbook on com-putational complexity.)

When analyzing Example 7, we notice that the third component is onlyneeded to “hide” the path which is created by the second component withina list of further edges, so that a simulating logspace Turing machine needsto be able to make nondeterministic choices. Therefore, by making use of asimilar construction, hardness ofPC2GSTT for the class L, i.e., determin-istic logspace, can be shown.

Corollary 25 PC2GSTT andPC2GSTTfs are hard forL. �Unfortunately, we do not know whether any of these two classes is con-

tainedwithin L.Wealsodonot knowcontainment in superclassesof Lwhichare subclasses of NL (e.g., so-called symmetric logspace or unambiguouslogspace), see, e.g., [25].

Now, we turn to the non-emptiness problem.


Theorem 26 For eachn ∈ N, the non-emptiness problem forPCnGSTTand forPCnGSTTfs grammar systems is complete forNL.

Proof. Since the non-emptiness problem is complete for the right-lineargrammars which coincide withPC1GSTT = PC1GSTTfs in our notation,the claimed NL-hardness follows immediately.

We need to show how an instance of the non-emptiness problem can besolved on a nondeterministic Turing machine with logarithmic work tape.We only sketch this in the following.

LetΓ = (N,K,Σ,G1, . . . , Gn) be aPCnGSTTfs. L(Γ ) �= ∅ iff thereare communication pointsk1, . . . , kr ∈ K+ such that, for some terminalstringsw1, . . . , wr+1 and nonterminalsA0, . . . , Ar of themaster (whereA0is the master’s axiom),

A0∗⇒ w1k1A1

∗⇒ w2k2A2∗⇒ . . .

∗⇒ wrkrAr∗⇒ wr+1

is a valid derivation (of the master grammar). We assume that “interme-diately” (i.e., in derivation steps indicated by

∗⇒), there are no communi-cations. This validity of course depends on the derivability of strings in acertain number of derivation steps in the called components. LetKi ⊆ K bethe set of components called, possible more than once, throughki. Clearly,the derivation sketched above is valid if all called components in each of thesetsKi yields some terminal string. The simulating nondeterministic Turingmachine proceeds as follows:REPEAT FOREVER

Guess whether there will be a next communication point.IF not: Verify non-emptiness as for right-linear grammars (only con-

sider the master component); in parallel: verify that all other components†may make at least as many derivation steps as the simulated master compo-nent did. IF successful, answer: YES (to the non-emptiness question).

IF so: Simulate in parallel all components, i.e., start with all compo-nents with the start symbols, and then repeatedly update all nonterminals ofalln components† according to the grammar rules. In sucha simulation, onlythe nonterminal symbols need to be recorded and not the terminal strings.Of course, the communication points are also of interest. If, in the courseof such a simulation, the master introduces someki ∈ K+, then all queriedcomponents must be able to generate a terminal string at this point.END REPEAT

Observe that a dagger symbol† indicates a place where it is necessarythat we fix the number of components in order to stay in NL, i.e., treat thenon-emptiness problem forPCnGSTT and not forPC∗GSTT. �Question: Is non-emptiness forPC∗GSTTfs hard for P? Observe that non-emptiness is even PSPACE-complete forMAT ∗(CF), see [16, Fig. 1].

528 H. Fernau

Acknowledgement.We thank Markus Holzer for discussions regarding this section.

7 Concluding remarks 1

We introduced a new class of right-linear PCGS which fit nicely into previ-ously defined language models (it lies between PRLL languages and matrixlanguages of finite index) and possesses attractive language theoretic prop-erties. Anyhow, many things remain to be clarified from a formal languagepoint of view. We mention, especially, the exact relations to obviously re-lated (but probably incomparable) classes like right-linear simple matrixlanguages and regular multiplicative valence languages. Valence languageshave been introduced in a grammar setting in [29]. They are equivalent toso-called regular unordered vector languages [11] and can be characterizedby blind multicounter automata, see [19,21,24].

{ anbn | n ≥ 0 }∗ is an example of a PCGSTTfs language (see Exam-ple 13) which is neither a right-linear simple matrix language nor a regularvalence language, see [23,24]. On the other hand, we conjecture that theright-linear simple matrix language{wh(w) | w ∈ {a, b}∗ }, whereh, themorphism defined byh(a) = b andh(b) = a, is not a PCGSTTfs languageand that the regular valence language{ anbmanbm | n,m > 0 } is not aPCGSTTfs language.

Moreover, decidability questions have been tackled only superficially.We just mention here that a result of Wood [48] can be sharpened by

stating that each2-parallel right-linear language is a regular additive valencelanguage or, equivalently, a language acceptable by a nondeterministic finiteautomaton with one blind counter [21]. That result can be generalized to thefollowing fact which we state without proof:

Theorem 27 R∗ is contained in the class of regular multiplicative valencelanguages. �On the other hand, we remark that the family of languages acceptable bynondeterministic finite automata equipped with one blind counter which isallowed to make at most one turn (i.e., one change between an incrementingand a decrementing phase), a family which can be characterized as the leasttrio containing{anbn | n ≥ 0}, see [8], is included in the family of2-parallel languages. The simulation itself is rather straightforward; the “firstgrammar” has only to memorize its target state (which is reached exactlyat the point where the simulated grammar [or automaton] makes its turn)which is the interface to the “second grammar” working in parallel (hence,simulating the counter applications).

The most important language theoretic issue would be to develop cri-teria for proving non-containment of certain languages inPC∗GSTT andPC∗GSTTfs.


Let us mention, finally, that it is quite easy to develop “analyzing” orautomata-theoretic models equivalent, in particular, toPC∗GSTTfs (also,in the case when generalizing those systems by allowing arbitrary context-free components), a task which has been proved to be quite hard for the“classical” PCGS definition, see [7].

Part 2: Language identification

8 Machine learning

The topic of Machine Learning possesses an ever-growing field of applica-tions, in particular triggeredwith the advent of the internet as amassmediumand the need to collect pieces of information automatically, leading to newlycoined catchwords like “data mining” and “discovery science”.

There are various models of machine learning around, i.e., ways to for-malize an automatic induction process. The simplest one is certainly “iden-tification in the limit from positive samples”, which was introduced byGold in 1967 [20] and further studied by Blum & Blum in [6] and An-gluin [2]. The identification machine IM gets an infinite stream of words(w1, w2, . . .) (possiblywith repetitions) from the (unknown) target languageL = {w1, w2, . . .} and outputs a hypothesis device (grammar or automa-ton in most cases)DM after having received the wordsw1, . . . , wM . L isidentified by IM if the sequence of devicesD1, D2, . . . converges to someD with L(D) = L in the discrete topological space of devices, i.e., there isa constantn (depending on the enumeration ofL) such that for alli ≥ n,Di = Dn. IM is called alearner for a language classL iff IM identifies anylanguage fromL correctly and independent of its enumeration order (whichmay also contain repetitions).

Unfortunately, the learning model is very weak from a formal languagepoint of view. For example, Gold has shown that there is no learner for a“superfinite” language class, i.e., a class containing all finite and at leastone infinite language. This seems to exclude any reasonable class of formallanguages, since nearly all “nice” language properties concerning closureoperations are lost. In particular, all classes of the Chomsky hierarchy arenot identifiable.

From a philosophical viewpoint, the non-learnability of any superfinitelanguage class is not such bad news: obviously, for the task of induction,it is necessary to generalize from some given finite sample set towards aconcept, i.e., an infinite language. This means that any finite sublanguage ofthat concept which contains the given finite sample set (which is also calleda “characteristic sample (set)” of the concept) cannot be learned, since itwould immediately be generalized to the (hopefully intended) concept.

530 H. Fernau

The weakness of themodel results, in particular, from four facts: (1) Thesuch-defined learning algorithms are deterministic (i.e., they do not containstochastic elements), (2) exact (not approximate) identification is required,(3) no further information (like “negative samples”) is given, and (4) the IMis “passive” and cannot ask questions to a “teacher” on its own. On the otherhand, the first three points make the language classes amenable to analysisusing tools from formal language theory.

9 Identification of PCGS languages

One way to show identifiability is to transfer known results on learning byformal language constructs. This has been successfully undertaken by usingcontrol languages [13,18,44,45] and certain context conditions [17].We tryto pursue a similar method in the case of PCGSTT here.

In order to explain the difficulties that arise in this approach when takingthe “classical” PCGSdefinition, let us describea typical identifiable subclassof regular languages.

9.1 Terminal distinguishable right-linear languages

Let x ∈ Σ∗. Then, defineTer(x) = {a ∈ Σ | uav = x andu, v ∈ Σ∗}andTer(L) =

⋃w∈L Ter(w).

Radhakrishnan and Nagaraja gave the following algebraic definition ofthe language class TDRL:

Definition 28 A regular languageL ⊆ Σ∗ is calledterminal distinguish-able right-lineariff

∀u, v, w, z ∈ Σ∗ : Ter(w) = Ter(z), {uw, vw} ⊆ L→ (uz ∈ L↔ vz ∈ L).

The corresponding language class is denoted byTDRL.

For our purposes, the following grammar-based definition is more ap-propriate.

Definition 29 LetG = (N,Σ, P, S) be a right-linear grammar with

P ⊆ (N \ {S}) × (Σ(N \ {S})) ∪ {S} × (N \ {S}).

G is calledterminal distinguishable right-linear, orTDRL for short, if it hasthe following properties:1.G is backward deterministic. (B → w andC → w impliesB = C.)2. For all A ∈ N \ {S} and for allx, y ∈ L(A), Ter(x) = Ter(y) holds.


(WriteTer(A) for short.)3. If (a)S → B andS → C are inP withB �= C orif (b)A→ aB andA→ aC are inP withB �= C, thenTer(B) �= Ter(C).

It has been shown in [13] based on [14]:

Theorem 30 A language isTDRL for short iff there is aTDRL grammargenerating it. �

In actual fact,RadhakrishnanandNagaraja [33,34] hadalreadyclaimedasimilar grammatical characterization for TDRL, but they gave no proof. Ourgrammatical definition of TDRL mainly adds point 3(a) to their definition,but this turns out to be an essential point in the equivalence proof. For furtherdiscussions, we refer to [13].

How does an inference algorithm for TDRL work?We need some auxil-iary notions to explain its behaviour. The algorithm receives an input samplesetS+ = {w1, . . . , wM}. Letwi = ai1 . . . aini ,whereaij ∈ Σ,1 ≤ i ≤M ,1 ≤ j ≤ ni. Theskeletal grammarfor the sample set is defined as

GS+ = (NS+ ∪ {S}, Σ, PS+ , S), where

NS+ = {Nij | 1 ≤ i ≤M, 1 ≤ j ≤ ni } and

PS+ = {S → Ni1 | 1 ≤ i ≤M }∪ {Nij → aijNi,j+1 | 1 ≤ i ≤M, 1 ≤ j < ni }∪ {Nini → aini | 1 ≤ i ≤M }.

The frontier stringof Nij is defined asFS(Nij) = aij . . . aini . This meansthatL(Nij) = {FS(Nij)}. Thehead stringofNij is defined by the equationHS(Nij)FS(Nij) = wi, i.e.,HS(Nij) = ai1 . . . ai,j−1.

Now, consider to nonterminalsNi,j andNk,l of a right-linear skeletalgrammar as TDRL-equivalent, denoted byNi,j ≡TDRL Nk,l, iff ≡TDRL=�+

TDRL, i.e., the transitive closure of the reflexive symmetric relation�TDRL, which is given byNi,j �TDRL Nk,l iff

1. FS(Ni,j) = FS(Nk,l) andTer(HS(Ni,j) = Ter(HS(Nk,l)), or2. HS(Ni,j) = HS(Nk,l).

Without formal proof, we state:

Lemma 31 ≡TDRL is an equivalence relation on the set of nonterminalsof the skeletal grammarGS+ . �

Now, define the grammarG as having nonterminals[N ]TDRL, where[N ]TDRL is thesetof thosenonterminalsofGS+ whichareTDRL-equivalenttoN , and the following rules:

initial rules S → [Ni,1]TDRL for some1 ≤ i ≤M ;

532 H. Fernau

transition rules [Ni,j ]TDRL → ai,j [Ni,j+1]TDRL for some1 ≤ i ≤ M ,1 ≤ j < ni; and

terminal rules [Ni,ni ]TDRL → ai,ni for some1 ≤ i ≤M .

SinceG is reduced,G is isomorphic to the canonical objects for TDRLdefined in [13,14] in terms of product automata. Hence, the sketched pro-cedure is indeed an identification algorithm for TDRL.

Asexhibited in [14] (basedon the implementation of the 0-reversible lan-guage identification algorithm due to Angluin [3]), the algorithm sketchedabove can be implemented to run in nearly linear time, i.e., in timeO(α−1(2�n)2�n), where� is the size of the input alphabet (this will be-come somewhat important in the following investigations),n is the totalinput size andα−1 is the inverse of the Ackermann function as defined byTarjan [46].

Example 32The skeletal grammar ofS+ = {w1 = ab, w2 = aab} is:

GS+ = ({S,N11, N12, N21, N22, N23, N24}, {a, b},{S → N11, N11 → aN12, N12 → b,

S → N21, N21 → aN22, N22 → aN23, N23 → b}, S) .

∈ NS HS FS Ter(FS)N11 λ ab {a, b}N12 a b {b}N21 λ aab {a, b}N22 a ab {a, b}N23 aa b {b}

The reader may verify that the inferred grammar is

G = ({S,A,B}, {a, b}, {S → A,A→ aA,A→ aB,B → b}, S) ,

whereA = [N11]TDRL andB = [N12]TDRL. More precisely,A ={N11, N21, N22} andB = {N12, N23}.G generates{anb | n > 0}.

In addition, “structural information” is sometimes provided to the learn-ing algorithm (see [40] in the case of TDRL). It is argued that in applications,certain structural information is always known.

9.2 Structural information in PCGS

In the case of PCGS, typical structural information would concern the datacommunication. Having this information somewhat supplied, a natural ideawould be to “learn” every component language separately. The main diffi-culty in this approach lies in the fact that classical PCGS allow the commu-nication of “state information”, i.e., nonterminals, from one component to


another. It is not clear how to cope with these symbols without telling theIM beforehand which nonterminals will occur in the grammar, informationwhich is generally regarded as part of the inference task.

Therefore, we introduced a variant of right-linear PCGS to avoid thisdrawback.

10 Concerning learnability

WediscussPCGSTTwhosegrammar components are, in somesense, TDRLlanguages. To this end, we must include the information of where transmis-sion of terminal strings occurred explicitly within the sample words.

In order to define the inclusion of the transmission information rigor-ously, let us reconsider thedefinitionof aderivation stepbetween twoconfig-urations(x1, . . . , xn)and(y1, . . . , yn)of aPCGSTTΓ = (N,K,Σ,G1, ...,Gn). As in the previous sections, we concentrate on centralized returningPCGSTT. LetQ be the alphabet consisting of all symbols of the form[q],whereq ∈ K+ is a string of query symbols which occurs within the mas-ter’s rules ofΓ . We will consider the new terminal alphabetΣ ∪ Q in thefollowing.

Whenx1 contains no query symbol, the relations⇒Q and⇒Q′ we aregoing to define coincide with⇒. If x1 contains at least one occurrenceof query symbols, we obtain a difference. Letx1 be of the formx1 =zqi1qi2 . . . qitA, wherez ∈ Σ∗,A ∈ N ∪ {λ} andqil ∈ K, 1 ≤ l ≤ t.

In this case,y1 = z[qi1qi2 . . . qit ]xi1xi2 . . . xitA in thecaseof the relation⇒Q andy1 = z[qi1qi2 . . . qit ]A in the case of the relation⇒Q′ . The otherparts of thedefinitionof⇒Q and⇒Q′ areas in thedefinitionof⇒, especiallyconcerning the differences between the fully synchronized and normalmodeof derivation.

Definition 33 Let Γ = (N,K,Σ,G1, ..., Gn) be a PCGSTT with mastergrammarG1 and let(S1, ..., Sn) denote the initial configuration ofΓ . TheQ-languagegenerated by the PCGSTTΓ is

LQ(Γ ) = {α1 ∈ (Σ ∪Q)∗ | (S1, . . . , Sn) ⇒∗Q (α1, . . . , αn)}.

TheQ′-languagegenerated by the PCGSTTΓ is

LQ′(Γ ) = {α1 ∈ (Σ ∪Q)∗ | (S1, . . . , Sn) ⇒∗Q′ (α1, . . . , αn)}.

Therefore, an idea would be to design an inference machine forLQ(Γ )instead ofL(Γ ). In this way, we formalize what we mean by explicit infor-mation about the communication structure. Since the contributions of thenon-master components cannot be uniquely reconstructed from this repre-sentation, we concentrate on what we call even PCGSTT. A PCGSTT is

534 H. Fernau

calledevenif every right-hand sideα of every rule in the PCGSTT containsexactly one symbol from the terminal alphabetΣ.

Consider the following example:

Example 34Γ = ({S1, S2, S3}, {q2, q3}, {a, b}, G1, G2, G3), whereG1contains the rulesS1 → aS1, S1 → aq2q3,G2 contains the rulesS2 → bS2andS2 → b, andG3 contains the rulesS3 → aS3 andS3 → a. Obviously,Γ is an even PCGSTT. Considered in the full synchronization mode,Γgenerates

L(Γ ) = {ambmam | m > 0}.Moreover, we find that

LQ(Γ ) = {am[q2q3]bmam | m > 0} and

LQ′(Γ ) = {am[q2q3] | m > 0}.

Given some wordamq2q3bmam in the example above, it is quite easy to tell

what the exact contributions of the non-master components are, sinceΓ waseven and worked fully synchronized; namely, since the word derived by themaster component up to the first query situation has lengthm, each of thetwo words queried from components 2 and 3 must have lengthm, too. Thismeans thatbm has been contributed by the second component and the postfixam has been contributed by the third component. Moreover, it is easy to seethat the wordam[q2q3] fromLQ′(Γ ) “corresponding” toam[q2q3]bmam inLQ(Γ ) can be deduced in this fashion, as well. Observe thatLQ′ collectsthe genuine contributions of the master grammar.

In general, we obtain the following decomposition algorithm.

Algorithm 35 (Decomposition algorithm)Let w = v1[k1]v2[k2] . . . v�[k�]v�+1 be in theQ-language of some evenPCGSTTfs, wherevi ∈ Σ∗ and ki ∈ K+; more precisely, letki =qi,1 . . . qi,ji , with qi,j ∈ K. Thenv2 can be decomposed asv2 = v1,1 . . .v1,j1v

′2, where|v1| = |v1,1| = . . . = |v1,j1 |, sincev1,m is the contribu-

tion of themth component queried by the master in stepm; therefore,|v1,m| = |v1|. Similarly, v3 can be decomposed asv3 = v2,1 . . . v2,j2v

′3,

where|v′2| = |v2,1| = . . . = |v2,j1 | and so forth. Finally,v�+1 is a part

which is derived by the master grammar only (without further queries toother components).

This decomposition allows us to use the following inference algorithm(based here on the inferrability of TDRL; of course, any inferrable regu-lar language subclass could be “plugged in” instead) for the class which


we are going to call TDRL− PCnGSTTfs.3 More precisely, let TDRL−PCnGSTTfs denote the class of languages which will be output by the in-ference algorithm given below. The corresponding languages contain querysymbols as guidance for the inference procedure.

Algorithm 36 (Inference of TDRL − PCnGSTTfs)1. Every input wordw can be decomposed in order to obtain sets of wordsWi generated by componenti.

2. Wi can be given to a TDRL algorithm for theith component.

Wehave tobeabit carefulwith themaster component, due to thepresenceof query symbols in words fromLQ′ .

We suggest the following variant for coping with query symbols:A word

wi = ai,1 . . . ai,i1 [ki,1]ai,i1+1 . . . ai,i2 [ki,2] . . . [ki,�]ai,i�+1 . . . ai,i�+1

fromLQ′ with ai,j ∈ Σ andki,j ∈ K+ can be generated by using the rules

• S → Ni,1,• Ni,j → ai,jNi,j+1 for j �= iν , 1 ≤ ν ≤ �+ 1,• Ni,iν → ai,iνki,νNi,iν+1 for 1 ≤ ν ≤ �, and• Ni,i�+1 → ai,i�+1 .

Themerging conditions are as before, now considering “symbols” which arein fact words fromΣK∗. This means, in particular, that the “alphabet” onehas to consider is not fixed beforehand but may keep on growing for sometime when new examples (with new sequences of query symbols) emerge.

Let us explore this phenomenon a bit more formally. Consider somelanguageLQ′ . Again, letQ be the alphabet consisting of all symbols ofthe form [q], whereq ∈ K+ is a string of query symbols which occurswithin the master’s rules ofΓ . Let Q = Q ∪ {λ}. Consider[ΣQ] = ΣQas a new alphabet. Letters from[ΣQ] will be written [aq] with a ∈ Σ andq ∈ Q. The homomorphismh : [ΣQ]∗ → (Σ ∪ Q)∗ shall be defined asthe trivial decodingh([aq]) = a[q] for a ∈ Σ and symbolsq from Q andh([a]) = a for a ∈ Σ. Then,LQ′ will be inferred by the suggested algorithmiff h−1(LQ′) ⊆ [ΣQ]+ is a TDRL language.

Let us continue our example in order to clarify our ideas:Consider as inputwordsof the inferencealgorithmaq2q3baandaaq2q3bb

aa.This means that the words[aq2q3] and[a][aq2q3] are passed to the TDRL

inference algorithm of themaster grammar. That inference algorithmwould3 Let us point the reader to a subtle minor point we will not discuss further: The TDRL

grammars which are inferred by the algorithm given above are not “even” according to ourPCGSTT definition, since the start symbol plays an extra role. However, this does not affectthe decomposition algorithm just described and may, hence, be neglected.

536 H. Fernau

yield S1 → N1,1,1, N1,1,1 → aN1,1,1 andN1,1,1 → aq2q3. (We use anadditional index in the nonterminals to distinguish the different grammarcomponents.)

Similarly, the wordsb andbb are given to the TDRL inference algorithmof the second component, yielding the rulesS2 → N2,1,1,N2,1,1 → bN2,1,1andN2,1,1 → b.

Analogously, the TDRL inference algorithm for the third componentoutputs the rulesS3 → N3,1,1,N3,1,1 → aN3,1,1 andN3,1,1 → a.

In this way, a correct PCGSTTfs is induced.4

Is there away of characterizing the language class TDRL−PC∗GSTTfssomehow? It is tempting to assume thatLQ ∈ TDRL − PC∗GSTTfs iffthe “component languages” (as they could be defined by the decompositionalgorithm given above) lie in TDRL. There is one subtle thing that thisconjecture might overlook: It is possible that not all words of the languagegenerated by a non-master component are actually queried by the master;more precisely, we have to single out the length of words which could bequeried by the master. LetMi be the set of lengths of words generated bytheith component which can be queried by the master.

In order to understand this difficulty, consider the following example:

Example 37Γ = ({S1, S′1, S2, S3}, {q2, q3}, {a, b}, G1, G2, G3)whereG1

contains the rulesS1 → aS′1, S

′1 → aS1 andS′

1 → aq2q3,G2 contains therulesS2 → bS2 andS2 → b, andG3 contains the rulesS3 → aS3 andS3 → a. Obviously,Γ is an even PCGSTTfs. We have

L(Γ ) = {a2nb2na2n | n > 0} .The inference algorithm would yield —after the trivial conversion into theeven PCGSTTfs form discussed in the footnotes above— the system

Γ ′ = ({S1, S′1, S2, S

′2, S3, S

′3}, {q2, q3}, {a, b}, G1, G2, G3),

whereG1 contains the rulesS1 → aS′1, S

′1 → aS1 andS′

1 → aq2q3, G2contains the rulesS2 → bS′

2, S′2 → bS2 andS′

2 → b, andG3 containsthe rulesS3 → aS′

3, S′3 → aS3 andS′

3 → a. Obviously,Γ ′ is an evenPCGSTTfs which is equivalent toΓ .

Nevertheless, we can state:

Theorem 38 Fix n ∈ N. LQ ∈ TDRL − PCnGSTTfs iff• h−1(LQ′) ∈ TDRL, and• for all 2 ≤ i ≤ n, there exists a languageLi ∈ TDRL such that

Li ∩ {w ∈ Σ∗ | |w| ∈Mi }is the language queried by the master component.

4 It is trivial to convert this induced PCGSTTfs into an equivalent even PCGSTTfs.


Proof. Consider a grammar system which is output by the identificationalgorithm. Obviously, theLQ language of such a system is of the formrequired by the theorem.

On the other hand, consider a languageLQ satisfying the above require-ments. Observe that the length setsMi can be computed from the words inLQ. Now, leth−1(LQ′) be enumerated as input of the TDRL-identificationalgorithm of the master component. Obviously, sinceh−1(LQ′) ∈ TDRLand due to the identifiability of TDRL, there will be some time stepn1,0 ofthis algorithm from which on the hypothesis grammar for the master com-ponent will not change any more and the corresponding generated languageequalsh−1(LQ′).

Now, consider some enumeration ofL′i = Li ∩ {w ∈ Σ∗ | |w| ∈

Mi } to the TDRL identification algorithm (of theith component). Due to[14, Lemma 9], the finite automata sequence corresponding to the outputsequence of right-linear grammars evoked by the enumeration ofL′

i consistssolely of sub-automata of the canonical finite automatonA of Li (where“canonical” refers to the definitions in [13,14]). Since the identificationalgorithm is conservative in the sense that it does not change its hypothesisas long as it is consistent with all the enumerated samples, and since thesearch space (the subautomata ofA) is finite, the algorithm will converge,i.e., the output will be constant froma certain time stepni,0 onward, yieldingsomeLi satisfyingL′

i = Li ∩ {w ∈ Σ∗ | |w| ∈ Mi }. In conclusion, theproposed identification algorithm will converge, and the obtained TDRL−PCnGSTTfs system generatesLQ. �Corollary 39 For eachn ≥ 1, the classTDRL − PCnGSTTfs is identifi-able in the limit from positive samples. �

Let us mention that the canonical objects required to define the conver-gence of the identification algorithm properly can be easily derived from theprevious theorem and the canonical objects for TDRL-languages, as exhib-ited in [13,14]. Moreover, in this way, suitable characteristic samples forlanguages in TDRL− PCnGSTTfs can be obtained.

It is easy to see that in this way, one extends the class of identifiable lan-guages (compared to the basic class TDRL) considerably at the cost of pro-viding the additional transmission information contained inLQ. Of course,one can consider “standardized” or uniform transmission information. Forexample, as suggested by Theorems 17 or 18, one can define learnable sub-classes ofR∗.

11 Concluding remarks 2

In the second part of this paper, we discussed identifiability of languageclasses defined via right-linear PCGS with terminal transmission. We fo-

538 H. Fernau

cussed on the notion of terminal distinguishability. Naturally, similar resultscan be obtained based onf -distinguishable languages for some arbitrary so-called distinguishing functionf , see [13]. Referring to the formal languageproblems discussed in the first section, we mention that these questionscould be discussed for subclasses such as TDRL− PC∗GSTTfs, as well.

Of course, the practical applicability of the sketched learning approachhas to be verified. Here, it is natural to make experiments with identifiablesubclasses of the regular languages different from TDRL, as well.

We only sketch one possible application scenario, motivated by discus-sion with people working in the hardware manufacturing industry:

Consider some hardware where some central unit collects its own er-ror protocol, as well as the protocols of other components. Based on thesecommon error protocols of the whole machine (each common error proto-col corresponds to one run of the machinery), an engineer has to extractthe “typical errors” and express these as regular expressions (for each ofthe hardware units). Assuming the different hardware components to befinite automata (which is, indeed, reasonable from a practical viewpoint)and assuming, further, that the protocols allow for distinguishing the originof different error messages (as can be done in the formalization developedabove bymeans of the decomposition algorithm), some inference algorithmfor PCnGSTTfs can be used to aid the engineer in his/her task, wheren isthe total number of hardware components involved.

Acknowledgement.We thank the anonymous referee for careful remarks on the submittedversion of the paper.

References

1. K. Abrahamson, L. Cai, S. Gordon: A grammar characterization of logarithmic-spacecomputation. In: Gh. Paun, A. Salomaa (eds) New Trends in Formal Languages, Vol1218 of LNCS, pp 247–255. Berlin Heidelberg New York: Springer 1997

2. D. Angluin: Inductive inference of formal languages from positive data. Inf Control45: 117–135 (1980)

3. D. Angluin: Inference of reversible languages. J ACM 29(3): 741–765 (1982)4. J.-M. Autebert: Some results about centralized PC grammar system. Theor Comput

Sci 215: 383–398 (1999)5. J. Berstel: Transductions and Context-Free Languages, Vol 38 of LAMM. Stuttgart,

Teubner 19796. L. Blum, M. Blum: Toward a mathematical theory of inductive inference. Inf Control

28: 125–155 (1975)7. H. Bordihn, J. Dassow, G. Vaszil: Grammar systems as language analyzers and recur-

sively enumerable languages. In: G. Ciobanu, Gh. Paun (eds) Fundamentals of Compu-tation Theory, Proc. FCT’99, Vol 1684 of LNCS, pp 136–147. Berlin Heidelberg NewYork: Springer 1999

8. F.-J. Brandenburg: Intersections of some families of languages. In: Automata, Lan-guages and Programming, Proc. ICALP’86, Vol 226 of LNCS, pp 60–69. Berlin Hei-delberg New York: Springer 1986


9. L. Cai: The complexity of linear PCGS. Comput Artif Intell 15: 199–210 (1996)10. L. Cai: The computational complexity of PCGSwith regular components. In: J. Dassow,

G.Rozenberg, A.Salomaa (eds) Developments in LanguageTheory II; at the crossroadsofmathematics, computer scienceandbiology, pp209–219.Singapore:WorldScientific1996

11. J. Dassow, Gh. Paun: Regulated Rewriting in Formal Language Theory, Vol 18 ofEATCS Monographs in Theoretical Computer Science. Berlin Heidelberg New York:Springer 1989

12. S. Dumitrescu, Gh. Paun: On the power of parallel communicating grammar sys-tems with right-linear components. RAIRO Informatique theorique et Applica-tions/Theoretical Informatics and Applications 31(4): 331–354 (1997)

13. H. Fernau: Identifying terminal distinguishable languages. Submitted revised versionof the extended abstract which appeared in the Proc. AMAI 2000, seehttp://rutcor.rutgers.edu/˜amai/AcceptedCont.htm

14. H. Fernau: On learning function distinguishable languages. Technical Report WSI–2000–13, Universitat Tubingen (Germany), Wilhelm-Schickard-Institut fur Informatik,2000. A conference version will appear at ALT 2000

15. H. Fernau: PC grammar systems with terminal transmission. In: R. Freund, A. Kele-menova (eds) Proc. InternationalWorkshop onGrammarSystems, pp 229–252. SilesianUniversity at Opava, 2000

16. H. Fernau, M. Holzer: Conditional context-free languages of finite index. In: Gh. Paun,A. Salomaa (eds) New Trends in Formal Languages, Vol 1218 of LNCS, pp 10–26.Berlin Heidelberg New York: Springer 1997

17. H. Fernau, M. Holzer: External contextual and conditional languages. In: C. Martin-Vide, Gh. Paun (eds) Recent Topics in Mathematical and Computational Linguistics,pp 104–120. Bukarest: The Publishing House of the Romanian Academy, 2000

18. H. Fernau, J. M. Sempere: Permutations and control sets for learning non-regularlanguage families. In: A.L. Oliveira (ed.) Proc. 5th International Colloquium on Gram-matical Inference ICGI 2000, Vol 1891 of LNCS/LNAI, pp 75–88. Berlin HeidelbergNew York: Springer 2000

19. H. Fernau, R. Stiebe: Sequential grammars and automata with valences. Manuscript.20. E. M. Gold: Language identification in the limit. Inf Control 10: 447–474 (1967)21. S. Greibach: Remarks on blind and partially blind one-way multicounter machines.

Theor Comput Sci 7: 311–324 (1978)22. M.Holzer: Multi-head finite automata: data-independent versusdata-dependent compu-

tations. In: I. Prıvara, P. Ruzicka (eds) Mathematical Foundations of Computer ScienceMFCS’97, Vol 1295 of LNCS, pp 299–308. Berlin Heidelberg New York: Springer1997

23. O. Ibarra: Simple matrix languages. Inf Control 17: 359–394 (1970)24. O.H. Ibarra, S. K. Sahni, C. E. Kim: Finite automatawithmultiplication. TheorComput

Sci 2: 271–296 (1976)25. K.-J. Lange: Complexity and structure in formal language theory. Fund Inform 25(3):

327–352 (1996)26. K.-J. Lange, R. Niedermeier: Data independence of read, write and control structures

in PRAM computations. J Comput Syst Sci 60: 109–144, 2000.27. H. A. Maurer, W. Kuich: Tuple languages. In: W. D. Itzfeld (ed.) Proc. ACM Interna-

tional Computing Symposium, pp 882–891. German Chapter of the ACM 197028. Gh. Paun: On the family of finite index matrix languages. J Computer Syst Sci 18(3):

267–280 (1979)29. Gh. Paun: A new generative device: valence grammars. Rev. Roumaine Math. Pures

Appl. XXV(6): 911–924 (1980)

540 H. Fernau

30. Gh. Paun: On the synchronization in parallel communicating grammar systems. ActaInf 30: 351–367 (1993)

31. Gh. Paun, L. Santean: Parallel communicating grammar systems: the regular case. Ann.Univ. Bucharest, Ser. Matem.-Inform. 38(2): 55–63 (1989)

32. H. Petersen: The head hierarchy for oblivious finite automata with polynomial advicecollapses. In: L. Brim, J. Gruska, J. Zlatuska, editors, Mathematical Foundations ofComputer Science MFCS’98, Vol 1450 of LNCS, pp 296–304. Berlin Heidelberg NewYork: Springer 1998

33. V. Radhakrishnan: Grammatical Inference from Positive Data: An Effective IntegratedApproach. PhD thesis, Department of Computer Science and Engineering, Indian In-stitute of Technology, Bombay (India), 1987

34. V. Radhakrishnan, G. Nagaraja: Inference of regular grammars via skeletons. IEEETrans Syst Man Cybern 17(6): 982–992 (1987)

35. R. D. Rosebrugh, D. Wood: A characterization theorem forn-parallel right linearlanguages. J Comput Syst Sci 7: 579–582 (1973)

36. R. D. Rosebrugh, D.Wood: Image theorems for simplematrix languages andn-parallellanguages. Math Syst Theory 8: 150–155 (1974)

37. R. D. Rosebrugh, D. Wood: Restricted parallelism and right linear grammars. UtilitasMathematica 7: 151–186 (1975)

38. P. Rossmanith: The owner concept for PRAMs. In: 8th Annual Symposium on Theo-retical Aspects of Computer Science STACS’91, Vol 480 of LNCS, pp 172–183. BerlinHeidelberg New York: Springer 1991

39. G.Rozenberg, D. Vermeir: On the effect of the finite index restriction on several familiesof grammars. Inf Control 39: 284–302 (1978)

40. J. M. Sempere, G. Nagaraja: Learning a subclass of linear languages from positivestructural information. In: V. Honavar, G. Slutski (eds) Proc. 4th International Col-loquium on Grammatical Inference ICGI’98, Vol 1433 of LNCS/LNAI, pp 162–174.Berlin Heidelberg New York: Springer 1998

41. R. Siromoney: On equal matrix languages. Inf Control 14: 133–151 (1969)42. I. H. Sudborough: A note on tape-bounded complexity classes and linear context-free

languages. J ACM 22(4): 499–500 (1975)43. I. H. Sudborough: On tape-bounded complexity classes andmulti-head finite automata.

J Comput Syst Sci 10(1): 62–76 (1975)44. Y. Takada: Learningevenequalmatrix languagesbasedoncontrol sets. In: A.Nakamura

et al. (eds) Parallel Image Analysis, ICPIA’92, Vol 652 of LNCS, pp 274–289. BerlinHeidelberg New York: Springer 1992

45. Y. Takada: A hierarchy of language families learnable by regular language learning.Inf Comput 123: 138–145 (1995)

46. R. E. Tarjan: Efficiency of a good but not linear set union algorithm. J ACM 22(2):215–225 (1975)

47. D. Wood: Properties ofn-parallel finite state languages. Technical Report 73/4, Mc-Master University, Hamilton, Canada, 1973

48. D. Wood: Bounded parallelism and regular languages. In: G. Rozenberg, A. Salomaa(eds) L Systems, Vol 15 of LNCS, pp 292–301. Berlin Heidelberg New York: Springer1974

Documents

Parallel communicating grammar systems with terminal transmission