A Prolog-based natural language front-end system

New Generation Computing, 2 (1984) 91-99 OHMSHA,LTD. and Springer-Verlag

Short Notes �9 OHMSHA, LTD. 1984

A Prolog-Based Natural Language Front-End System

Hiroshi M A R U Y A M A * Science Institute, IBM Japan, Ltd., Sanban-cho, Chiyoda-ku, Tokyo 102, Japan Akinor i Y O N E Z A W A Department of Information Science, Tokyo Institute of Technology, Ookayama, Meguro-ku, Tokyo 152, Japan

Received 18 November 1983 Revised manuscript received 13 January 1984

Abstract A Prolog-based natural language front-end system is described with the following major issues of discussion : Domain independence of the syntax analyser was achieved by the 'generate-and-test' notion and the domain independent semantic representation; Determiners were treated as higher order predicates; A technique called 'syntactic feature' was employed to write a readable parser in Prolog.

Keywards: Natural Language Processing, Language Parsing and Understanding, Definite Clause Grammar, Domain Independence, Quantification, Syntactic Features.

w 1 Introduction Prolog's simple but powerful semantics have recently become the center of

attention in natural language processing. Definite Clause Grammar (Pereira & Warren8)), which is a syntactic sugar of Prolog, proved its clarity and power in natural language understanding as a vehicle for sentence parsing in the phrase structure grammar framework.

The motivation of our study is to explore the possibility of using Prolog in writing a practical natural language front-end system. We first started with a small, 100 line, English question answering system written in Prolog. This experience confirmed the usefulness of Prolog. From this, we rebuilt a larger version of the system, in light of portability and maintainability.

This paper presents the ideas we obtained from the larger version of our system. We will first describe the design of the system, and then discuss the following issues which we were most concerned with in implementation ; (1) domain independence of the syntax analyser, (2) treatment of determiners, and (3) the technique of

* The work reported here was done while the author was at the Tokyo Institute of Technology, Dept. of Information Science.

92 H. Maruyama, and A. Yonezawa

'syntactic features' for writing a readable parser in Prolog. Finally, we will discuss the usability of Prolog in natural language understanding.

Two successful works in the same direction are well known, one by McCord, 6) which influenced our system considerably, and the other, Warren and Pereira's work 9) which was independently carried out. The differences between our system and these two systems will also be discussed in the appropriate sections.

w Experimental Sys t em Des ign

2 . 1 The System Structure A natural language front-end has to incorporate semantic knowledge as well

as syntactic knowledge. While the semantics is domain dependent, the syntax is not, but rather specific to a particular language. In earlier systems, the syntax analysers utilized domain specific knowledge in the early stages of the syntax analysis processes in order to reduce ambiguity. Therefore, these syntax analysers heavily incorporated domain specific portions and could not be reused in other applications in spite of many efforts which were spent on writing the code of such syntax analysers.

Our major concern with the system design is to make syntax analysers as domain-independent as possible to retain portability. If the knowledge of the language is packed in the syntax analyser, and it doesn't contain any domain specific part, it will be applicable to other domains.

Our strategy for realizing this goal is to use the "'old" notion of "generate and test", 1) which is very natural to Prolog. The whole of our system consists of two independent components: the syntax analyser and the semantic analyser. When an input sentence is syntactically ambiguous, the syntax analyser will generate one candidate of the possible syntactic interpretation at a time, and the semantic analyser will test whether that syntactic interpretation makes any sense in a specific domain or not. The relationship between the syntax analyser and the semantic analyser is shown more clearly in Fig. 1.

Syntax analyser Parser Determiner processor

Word class ] dictionary

Semantic ~ Inference analyser system

and database

Fig. 1 The system structure.

The syntax analyser reads an English sentence and produces a syntax analysis result, which we call 'intermediate semantic representation'. The syntax analyser consists of the parser and the determiner processor.

The parser accesses a dictionary of the word class which contains only syntactic information of each word.

The semantic analyser transforms the intermediate semantic representation into

A Prolog-Based Natural Language Front-End System 93

a domain specific representation. If this process fails, then backtrack is invoked, and another intermediate semantic representation is produced by the syntax analyser, which is then semantically analysed. To do this, the semantic analyser accesses a database and an inference system of the selected domain.

One of the noteworthy differences of our system from the Warren's and McCord's systems is found in the dictionary contents, in both the Warren's system and McCord's system, each entry of the dict ionary contains domain specific representation of the word's meaning and the semantic markers of the relating objects, together with syntactic information. The portabil i ty of the system is maintained by localization of the domain specific knowledge only in the dictionary and by using some pattern matching techniques for sentence disambiguation. But the configuration of the semantic markers should be carefully designed so that their syntax analyser will produce only one correct parsing.

On the other hand, the dictionary in our system does not contain any domain specific knowledge and is again portable to various domains.

2 . 2 Intermediate Semantic Representation To achieve domain independence, the syntax analyser cannot use the knowl-

edge which is specific to a particuIar domain. But, on the other hand, the intermediate semantic representation must carry all the syntactic information that can be extracted from the input sentence. In addition, this representation must be well- defined so that anyone who knows the specification of this representation can write his/her own semantic analyser to build natural language front-ends using this syntax analyser.

Our representation is similar to a Prolog goal, thus it can be executed by a Prolog interpreter directly or with a slight modification. Functors and atomic symbols are extracted from the words of the input sentence. For example, the input sequences

"X likes Y." "'X of Y."

produce semantic representations such as : like(*x, *y, *c) ~ off*x, *y).

where *x and *y correspond to X and Y in the input sentence respectively, and *c is reserved for context information, i.e. to link with adjunct phrases such as adverbs or prepositions. The number of arguments of a verb-type predicate depends on its t ransi t ivi ty; i.e. how many objects and complements it requires. We introduce predicates ~is" and "hasprop' for nouns and adjectives. Such phrases as

"a man" "a red car"

are transformed into is(*x,man) is(*x,car) & hasprop(*x,red).

We do not employ predicates such as man(*x) or red(*x), because when a question like

"What color does the car have ?" is given, the corresponding representation including

t In this paper, we use "*" to represent Prolog variables_ Upper case letters are used for readability.


hasprop(CAR001, *x) can be directly executed by Prolog interpreter, while the one including

*x(CAR001). cannot.

Using these conventions, the syntax analyser can generate intermediate semantic representations without any domain specific knowledge. For example,

"The quick brown fox jumps over a lazy dog." can be translated into

def(*x, hasprop(*x, quick)&hasprop(*x,brown)&is(*x,fox)) & jump(*x,*c) & over(*c,*y) & hasprop(*y,lazy) & is(*y,dog)

using a word class dictionary with the following contents : isnoun(fox), isadj(quick). isnoun(dog), isadj(brown). isprep(over), isadj(lazy). isverb(jump, (transitivity)t).

To remove syntactic ambiguity, this intermediate semantic representation must be tested. For example, consider the following sentence :

"Time flies like an arrow." One of the possible interpretations of this sentence which are produced by our syntax analyser is

is(*x,time-fly) & like(*x, *y, *c) & is (*y, arrow), which is not correct. The semantic analyser should be built so that it signals a failure when *x, the first argument of 'like', appears in an "is' predicate which is inconsistent with

is(*x, human). Writing this kind of procedure for each predicate seems somewhat troublesome, but we believe that the number of necessary predicates are relatively small for many applications.

2 . 3 Determiners and Higher Order Meanings Some types of English sentences such as

"Every man likes a beer." o r

"What is on the third floor." have 'higher order meanings" in our representation. The intermediate semantic representations of the above examples are expressed as

imply(is(*x,man), like(*x, *y, *c)&is(*y, beer)) and

what(*x, on(*x, 3F)). The word "higher order' comes from the fact that predicates appear in another predicate as its arguments. Usually these kinds of situations appear with determiners (Dahl3)). The question is 'scoping' of such predicates ; i.e. which part of the syntax tree should be converted into arguments of the higher order predicate. Basically our solution is the same as McCord's. 6) The rules we used are shown in Fig. 2.

The major difference from McCord's system is the treatment of definite noun

~" Syntactic features of transitivity which will be discussed in section 2.4.


I.

2.

3.

Higher order predicates in a relative clause modifying a noun phrase cannot dominate the noun phrase. Higher order predicates appearing in a definite noun phrase, i.e. noun phrase with determiner "the" cannot dominate the noun phrase. As far as the above two conditions are satisfied, the scope of higher order predicates follows the precedence below, i.e. predicates with higher precedence take all of the predicates with lower precedence as arguments : a. interrogative phrase such as "'what", "how many", etc. (highest) b. noun phrase with determiner "'no" or "'all". c. plural definite noun phrase. d. indefinite noun phrase (noun phrase without "the"). e. other clauses, other phrases (except singular definite noun phrase). f. definite noun phrase (lowest).

Fig. 2 Scoping rules of higher order predicates.

phrases. When the determiner processor encounters a definite noun phrase, the semantic processor is called to find the referent of the noun phrase in a history stack or the database. If it is not found, the determiner processor signals failure of the parsing. If the definite noun phrase is plural, then the intermediate semantic representation produced is

imply(member(*x,Ethe referent resolved]),p(*x)).

For example, "The three beers are on the table."

is converted into imply(member( *x, [BEER001, BE ER002, BEER003] ), on(*x, TABLE00] )).

assuming the referent of "the three beers" and "the table" are [BEER001, BEER002, BEER003] and TABLE001, respectively.

Since these rules of 'higher order meanings' are considered to be domain independent, we decided to include this process in the syntax analyser, and not in the semantic analyser, This is the reason why the syntax analyser is split into two parts ; one which produces a syntax tree and the other which transforms this tree into an intermediate representation.

2 . 4 Syntactic Features We pointed out (in Section 2.1) that anyone who knows the specification of

the intermediate semantic representation can use our syntax analyser with a small amount of effort. In this section, we discuss how to write a clear, readable parser in Prolog. Natural language parsers based on phrase structure grammar must have some mechanism to carry context sensitive information from one node to another in a syntax tree. In Lisp-based systems, this function is usually implemented by tree traversals and value assignments to 'registers' attached to the parsing tree nodes. In Prolog-based systems, on other hand, it can be naturally represented by unifications of logical variables.

In spite of this advantage, the more sophisticated the grammar becomes, the more context sensitive information must be added and the number of arguments of each nonterminal symbol becomes larger ; what is worse, sometimes cut-operators are introduced in the parser to express complicated conditions. A Prolog program with a large number of arguments and cut-operators is not clear, and from our experience


in writing the English parser it is difficult to read and debug such a program. The solution we propose is to use only one argument for one terminal symbol,

which carries all context sensitive information. The argument, called a "syntactic feature', has a structure so that the needed portion of the information is easily accessible. For example, the structure of a syntactic feature of the nonterminal CLAUSE l~ is shown in Fig. 3. It reads, the type of a clause <*ctype> is either declarative, imperative, etc.. Each piece of information in the structure is determined anytime during the parsing, and once fixed, succeeding parsing will be done under that assumption until it fails.

< * f e a t u r e > : : = < * c t y p e > . < * i n f l > . < * t r a n s >

<*ctype> ::= DCL l IMP I YESNO [ WnJ ~SUBJ l WrISHORT I SEC. ADJUNCT [ SEC.(RSQ. nil) I SEC. RSNG

<*infl> ::= ROOT INF.<*number> NOTINF. TO NOTINF.(PART. EN) NOTINF.(PART. ING)

<*trans> ::= THERE <*objl>.<*comp>.<*obj2>.<*voice>

<*objl> ::= nil OBJI. NG OBJI.(RSNG.<*infI>)

<*comp> ::= nil COMP

<*obj2> ::= nil OBJ2. NG OBJ2.(RSNG.<*infI>)

<*voice> ::= ACTV PASV.<*prep>

<*number> ::= SING. FIRST- SING. SECOND SING. TIRD PL

~g. 3 The structure of the syntactic ~atures of CLAUSE.

Our parser is written in DCG, and each rule has the following form : <left-hand) , <test-part>, <right-hand I> . . . . . <right-hand,).

The "test-part' represents in what condition this rule should be applied. To illustrate


how it works, consider the following sentences : "1 like to drink." "1 like drinking." "1 want to drink." "I want drinking.'"

To reject the last sentence, we write the following rule of OBJ I, which accounts for a direct object of a verb :

OBJl(*objl) ,{*objl = [OBJI , [RSNG,*inf l ] ] , ctype(*cIause, [SEC,RSNG] ), infl(*clause,*infl)}, CLAUSE(*clause) .

CLAUSE(*clause) , {infl(*clause, *infl)},VP(*infl). VP(TO) ' [ to], {verb([ROOT, *trans])}, MODS(*trans). VP(ING) , {verb( l iNG, *trans])}, MODS(*trans).

The goal 'MODS' accounts for objects and complements o f the verb. The dict ionary entries of ' l ike' and "want' are shown in Fig. 4.

isverb(like,[[OBJl,*type],nil,nil,~ ) :- *type=NG [ *type=[RSNG,T~ [ *type=[RSNG,ING].

isverb(want,[[OBJl,*type],nil,nil,*]) :- * t y p e = N G [ * t y p e = [ R S N G , T ~ .

Fig. 4 The dictionary entry of "like" and 'want'.

Using this technique we could restrict accessibility of the context sensitive information to the necessary portion as well as the avoidance of the cut-operators.

w Evaluation The question answering system we developed was implemented on a small

Prolog interpreter which in turn is implemented on Lisp. The code is about 700 lines including the database and dictionary. Most inputs are processed within one to three seconds on M-200H processor. This performance is achieved with a dict ionary size of about 200 words in root form.

The parser is written in DCG slightly augmented with the idea o f the extraposition grammar (Pareira 7)) and the syntactic features discussed previously.

The domains we chose as a test bed for the syntax analyser were the blocks world, a small relational database and a telephone directory lookup system. Since these domains are small, the intermediate semantic representation produced by the syntax analyser is directly executed by the Prolog interpreter.

For the anaphora resolution, we used the simplest strategy, a history stack. Every time the referent of a noun phrase is resolved, it is recorded on the axiom store of the Prolog interpreter in the form of

history(*x).

w Conclusion A Prolog-based domain-independent syntax analyser system has been described

and some useful ideas have been presented. We first concluded that Prolog is really appreciable in building natural language front-end systems. This is due to not having to write any special parsing mechanisms nor any inference mechanism. In addition,


the 'generate and test' notion as well as 'syntactic features' which were essential for our implementation, can be naturally expressed in Prolog.

Besides these merits, the followings are further noteworthy comments on DCG and Prolog.

For other languages such as Japanese, the case grammar framework seems to be preferable to the usual phrase structure grammar framework which is the basic notion of DCG. Of course, it is possible to realize such a case grammar framework by use of DCG in a slot-filling fashion (McCord6)), however, it tends to require the use of rather tricky techniques. Besides, left recursive rules cannot be used in a top- down, depth-first parser. A Prolog-based bottom-up parser (Matsumoto et al. 5)) is proposed to overcome this drawback. Another problem of DCG is that there is no way o f knowing the reason for failure when the entire parsing process for a sentence fails. This is because backtracking destroys variable binding information. From the view point of human factor, it is fatal in the sense that the system cannot respond to the user other than "1 don ' t know. Try again", for such ill-formed sentences. Modulari ty is another problem of Prolog. For example, a short term memory is necessary in the anaphora resolution process to remember the discourse, for which we use axiom store instead. Also, using Prolog is not a good way to solve an inherently deterministic problem, because it takes some time to remove unnecessary backtracking and unification when debugging programs.

We aimed for an easily adaptable, easily maintainable natural language front- end by using DCG and Prolog. Refinement should be done on the semantic representation produced by the front-end, but we believe that in almost all applications the semantics of input sentence can be represented in logic programming frame work. We intentionally avoided the efficiency matter, but in our experience we did not feel that our approach was restricted by performance limitations. Technology of hardware is still growing rapidly, and when parallel processing architecture of logic programming becomes available, there will be more of a chance to use Prolog in writing more sophisticated natural language front-end systems.

References 1) Clocksin, W. F. and Mellish, C. S. : "Programming in Prolog" (Springer, 1981). 2) Colmerauer, A. :"Metamorphosis Grammars", in Bolc (ed.), Natural Language

Communication with Computers, Lecture Note in Computer Science, 63 (Springer- Verlag, 1978).

3) Dahl, V. : "Quantification in a Three-valued Logic for Natural Language Question- answering Systems", Proceeding of 6th International Joint Conference on Artificial Intelligence (Tokyo, 1979).

4) Maruyama, H. : "Natura l Language Understanding System Using Prolog", Pro- ceedings of The Logic Programming Conference '83 (1983) Ein Japanese].

5) Matsumoto, Y., Tanaka, H., Hirakawa, H., Miyoshi, H. and Yasukawa, H. : "BUP : A Bottom-Up Parser Embedded in Prolog', New Generation Computing, l (1983) 145-158.

6) McCord, M. C. : "Using Slots and Modifiers in Logic Grammars for Natural Lan- guage", Artificial Intelligence, 18 (1982) 327-367.

7) Pereira, F. : "Extraposition Grammars", American Journal of Computational Lin- guistics, 7 (1981) 243-256.

8) Pereira, F. C. N. and Warren, D. H. D. : "Definite Clause Grammars for Language


Analysis-- A Survey of the Formalism and a Comparison with Augmented Transition Networks", Artificial Intelligence, 13 (1980) 231-278.

9) Warren, D. H. D. and Pereira, F. C. N. : "An Efficient Easily Adaptable System for Interpreting Natural Language Queries", American Journal of Computational Lin- guistics, 8 (1982) 110-122.

10) Winograd, T. : "Understanding Natural Language" (Academic Press, New York, 1972).

Documents

A Prolog-based natural language front-end system