Center for PersonKommunikation P.1 CPK NLP Suite for Spoken Language Understanding ( tb/nlpsuite) Tom Brøndsted, CPK. Aalborg University

P.1

Center for PersonKommunikation

CPK NLP Suite for Spoken Language Understanding

(http://www.cpk.auc.dk/~tb/nlpsuite)

Tom Brøndsted, CPK. Aalborg [email protected]

P.2


Technical Details

• Programs for syntactic/semantic parsing, generation of language models …

• Reads/writes a number of grammar formats

• Free for research/non-commercial use

• Available in source (C++, lex/yacc)

• Compiles and runs on “any” machine under “any” OS having a 32 bit C++ compiler

P.3


25 ProgramsAPS PTR PSG ICM VOC

PARS

CONV

TREC

TSLU

SGEN

e.g. psgtrec(.exe)

(what is done with/to the grammar)

(grammar format)

P.4


Grammar Formats IAPS PTR PSG ICM VOC

PARS

CONV

TREC

TSLU

SGEN

Augmented Phrase Structure Grammer format based on a subset of the “EUROTRA User Language”

•Compound (but non-nested) features and structure sharing (“variables”)

•Semantic frames are created either

•through percolation of features

•with mapping rules creating frames when specified conditions are met

P.5


Grammar Formats IIAPS PTR PSG ICM VOC

PARS

CONV

TREC

TSLU

SGEN

grammar format based on PC-PATR/PATR II (Shieber etc.)

•Compound (including nested) features and structure sharing (“variables”)

•Semantic frames are created

•through percolation of features

•no mapping

P.6


Grammar Formats III-IV-VAPS PTR PSG ICM VOC

PARS

CONV

TREC

TSLU

SGEN

Simple symbol-based (non-unification based) formats including:

•Simple BNFs intended for teaching

•Finite state grammars intended for speech recognition

•[further details in the paper]

P.7


Program Types IAPS PTR PSG ICM VOC

PARS

CONV

TREC

TSLU

SGEN

Natural language parser

Input: 1-best/n-best. Output: semantic frames

•Unification-based left-corner chart parser

•Bottom-up, left-right with top-down filtering

•Exhaustive parsing

•API: allows interface to speech recogniser and dialogue management

P.8


Program Types IIAPS PTR PSG ICM VOC

PARS

CONV

TREC

TSLU

SGEN

Grammar converter

Input: NL-Grammer. Output: equivalent language model

•Types of language models: RTNs, FSNs, Word Pair, “No grams”

•can be written in the HTK standard lattice format etc.

•can substitute the network loading routine of a speech recogniser

P.9


Program Types III-IVAPS PTR PSG ICM VOC

PARS

CONV

TREC

TSLU

SGEN

Grammar constrained typed text recogniser, simulation of speech recognition/understanding

Input: typed sentence. Output: a “similar” sentence covered by the grammar (or corresponding frames)

•standard linear Viterbi pattern-matching algorithm

•uses FSN (“full-gram”) constraints

•supports “garbage models”

P.10


Program Types VAPS PTR PSG ICM VOC

PARS

CONV

TREC

TSLU

SGEN

Random sentence generator, exploring the coverage of grammars

Input: a NL-grammar. Output: a number of sentences covered by the grammar

•Uses a fully equivalent RTN derivation of the grammar

•Corresponds to the HTK-utility HSgen

P.11


ANSI C API (+JAVA)APS PTR PSG ICM VOC

PARS

CONV

TREC

TSLU

SGEN

Functions for

•Loading grammars

•Activating/deactivating subgrammars

•retrieving semantic frames from recognition results

•ANSI C structure implemented for semantic frames

P.12


Adding a NL Grammar FormatAPS PTR PSG ICM VOC XXX

PARS

CONV

TREC

TSLU

SGEN XXXlex/yacc

code

P.13


Adding a Recognition Grammar Format

APS PTR PSG ICM VOC

PARS

CONV

TREC

TSLU

SGEN

RTN

FSN

WPG

NOG

HTK write procedure

VOC write procedures

XXX write procedure (ca. 80 lines of code)

P.14


Conclusion

• Represents work in progress

• Available for non-commercial use “as is”

• Known inconveniences

• Users are welcome to extend the suite

P.15


Basic idea: Controlling speech understanding via one grammar

Speech recogniser

Natural languageparser

“sentence”

semantic frames

grammar

grammar

conversion

speech

P.16


Tutorial Example in Paper

GrapHvite(Entropic)

APSPARS

“sentence”

semantic frames

HTKstandard lattice

EUROTRAAPS Gr.

APSCONV

speech

P.17


Application Example

• Voice driven email application from the GrapHvite tutorial (Entropic)– A. Using the Entropic grammar tools (HTK and

GrapHvite tools) to make HTK standard lattice– B. Using the CPK NLP suite to implement the APS

grammar and the HTK standard lattice.

P.18


Using the HTK Grammar Tools

• An EBNF-like grammar definition language – HTK tools: Hparse, Hbuild, HSGen, HDMan

$digit = ONE | TWO | THREE | FOUR | FIVE | SIX | SEVEN | EIGHT | NINE | OH | ZERO;$name = [ JOOP ] JANSEN | [ JULIAN ] ODELL | [ DAVE ] OLLASON | [ PHIL ] WOODLAND | [ STEVE ] YOUNG;( SENT-START ( DIAL <$digit> | (PHONE|CALL) $name) SENT-END )

P.19


Using GrapHvite Grammar Tool

• Visual Netbuilder:

P.20


Using the CPK NLP suite

• Implementing the grammar in the APS format using a texteditor

• Testing the grammar

• Simulating speech recognition and understanding

• Converting the APS into HTK standard lattice

P.21


The APS formalism

BNF• Rules consist of symbols,

e.g. BritishCity, DanishCity

• Basic operation:– string comparison

– Boolean (true or false)

APS• Rules consist of feature sets,

e.g. {cat=city,country=UK}, {cat=city,country=DK}

• Basic operation:– unification

– Null or a new feature set

P.22


Semantics in the APS

• Simple: Percolation of values to the top node

• Mapping, postprocessing (see paper)

STRUCTURE BUILDING RULE{cat=person, emailaddr=$A} [{cat=lastname, emailaddr=$A}].

LEXICAL RULE{lex=brondsted,cat=lastname, [email protected]}.

P.23


the msg_spec subnetwork

Examples: delete message number three forward previous message go to first

Semantics: a) an absolute number b) a relative number

P.24


the msg_spec subnetwork

LEXICAL RULES{lex=previous,cat=spec, relpos=-1}.{lex=current, cat=spec, relpos=0}.{lex=this, cat=spec, relpos=0}.{lex=next, cat=spec, relpos=1}.

{lex=first, cat=spec, abspos=0}.{lex=last, cat=spec, abspos=10000}.

{lex=message,cat=msg}.{lex=email,cat=msg}.<number>.

P.25


Testing/running msg_spec

apspars msg_spec.aps msg_spec.snt

apstslu msg_spec.aps 0

apstrec msg_spec.aps 0

apssgen msg_spec.aps

apsconv -f -h msg_spec.aps

testing on a test database

testing random generation

simulating recognition

simulating speech understanding

fullgram HTK standard lattice

Documents

Center for PersonKommunikation P.1 CPK NLP Suite for Spoken Language Understanding ( tb/nlpsuite) Tom Brøndsted, CPK. Aalborg University