Upload
gerd
View
50
Download
0
Embed Size (px)
DESCRIPTION
Representing Meaning. Lecture 18 12 Sep 2007. Transition. First we did words (morphology) Then simple sequences of words Then we looked at true syntax Now we’re moving on to meaning. Where some would say we should have started to begin with. Meaning. - PowerPoint PPT Presentation
Citation preview
Representing Meaning
Lecture 1812 Sep 2007
Transition
First we did words (morphology) Then simple sequences of words Then we looked at true syntax Now we’re moving on to meaning. Where some would
say we should have started to begin with.
Meaning
Language is useful and amazing because it allows us to encode/decode… Descriptions of the world What we’re thinking What we think about what other people think
Don’t be fooled by how natural and easy it is… In particular, you never really… Utter word strings that match the world Say what you’re thinking Say what you think about what other people think
Meaning
You’re simply uttering linear sequences of words such that when other people read/hear and understand them they come to know what you think of the world.
Meaning Representations
We’re going to take the same basic approach to meaning that we took to syntax and morphology
We’re going to create representations of linguistic inputs that capture the meanings of those inputs.
But unlike parse trees and the like these representations aren’t primarily descriptions of the structure of the inputs…
In most cases, meaning representations are simultaneously descriptions of the meanings of utterances and of some potential state of affairs in some world.
Introduction
Meaning representation languages: capturing the meaning of linguistic utterances using formal notation so that they make semantic processing possible
Example: deciding what to order at a restaurant by reading a menu, giving advice about where to go for dinner Requires knowledge about food, its preparation, what people
like to eat and what restaurants are like Example: answering a question on an exam
Requires background knowledge about the topic of the question
Example: Learning to use a software by reading a manual Requires knowledge about current computers, the specific
software, similar software applications, knowledge about users in general.
Semantic Analysis
Semantic analysis: mapping between language and real life I have a car:1. First Order Logic∃ x,y: Having(x) ^ Haver(speaker,x) ^ HadThing(y,x) ^ Car(y)
2. Semantic Network
Speaker
Car
Having
Haver Had-thing
Speaker Car
3. ConceptualDependency
Diagram
POSS-BY
4. Frame BasedRepresentation
Having Haver: Speaker HadThing: Car
Semantic analysis
A meaning representation consists of structures composed from a set of symbols, or representational vocabulary.
Why meaning representations are needed? What they should do for us?
Example: Giving advice about restaurants to tourists.A computer system that accepts spoken language queries from tourists
and constructs appropriate responses by using a knowledge base of relevant domain knowledge.
Representations that Permit us to reason about their truth (relationship to some world) Permit us to answer questions based on their content Permit us to perform inference (answer questions and determine
the truth of things we don’t actually know)
Semantic Processing
Touchstone application is often question answering Can a machine answer questions involving the
meaning of some text or discourse? What kind of representations do we need to
mechanize that process?
Verifiability
Verifiability: Ability to compare the state of affairs described by a representation to the state of affairs in some world modeled in a knowledge base.
Example: Does Anarkali serve vegetarian food?
Knowledge base (KB) Sample entry in KB: Serves(Anarkali,Vegetarian Food) Convert question to logical form and verify its truth value
against the knowledge base
Unambiguousness
Example:I want to eat someplace near Chowringhee.(multiple interpretations)
Interpretation is important Preferred interpretations Regardless of ambiguity in the input, it is critical that a
meaning representation language support representations that have a single unambiguous interpretation.
Vagueness
Vagueness: I want to eat Italian food.- what particular food?
Meaning representation language must support some vagueness
Canonical form
Inputs that have the same meaning should have the same meaning representation.
Distinct sentences having the same meaning Does Anarkali have vegetarian dishes? Do they have vegetarian food at Anarkali? Are vegetarian dishes served at Anarkali? Does Anarkali serve vegetarian fare?
Words have different senses, multiple words may have the same sense Having vs. serving Food vs. fare vs. dishes (each is ambiguous but one sense of each
matches the others) Alternative syntactic analyses have related meaning (Ex: active vs
passive)
Inference and variables; expressiveness
Inference and variables: Can vegetarians eat at Anarkali? I’d like to find a restaurant that serves vegetarian food. Serves (x,VegetarianFood) System’s ability to draw valid conclusions based on the meaning
representations of inputs and its store of background knowledge. Expressiveness:
system must be able to handle a wide range of subject matter
Semantic Processing
We’re going to discuss 2 ways to attack this problem (just as we did with parsing) There’s the theoretically motivated correct and
complete approach… Computational/Compositional Semantics
And there are practical approaches that have some hope of being useful and successful.
Information extraction
Meaning Structure of Language
The various methods by which human languages convey meaning Form-meaning associations Word-order regularities Tense systems Conjunctions Quantifiers A fundamental predicate-argument structure
Asserts that specific relationships / dependencies hold among the concepts underlying the constituent words and phrases
The underlying structure permits the creation of a single composite meaning representation from the meanings of the various parts.
Predicate-argument structure
Sentences Syntactic argument framesI want Italian food. NP want NPI want to spend less than five dollars. NP want Inf-VPI want it to be close by here. NP want NP Inf-VPThe syntactic frames specify the number, position and syntactic category of the
arguments that are expected to accompany a verb.
Thematic roles: e.g. entity doing the wanting vs. entity that is wanted (linking surface arguments with the semantic=case roles)
Syntactic selection restrictions: I found to fly to Dallas. Semantic selection restrictions: The risotto wanted to spend less than ten
dollars. Make a reservation for this evening for a table for two persons at eight:
Reservation (Hearer,Today,8PM,2)
Any useful meaning representation language must be organized in a way that supports the specification of semantic predicate-argument structures. Variable arity predicate-argument structures The semantic labeling of arguments to predicates The statement of semantic constraints on the fillers of
argument roles
Model-theoretic semantics
Basic notions shared by representation schemesAbility to represent
Objects Properties of objects Relations among objects
A model is a formal construct that stands for the particular state of affairs in the world that we are trying to represent.
Expressions in a meaning representation language will be mapped in a systematic way to the elements of the model.
Vocabulary of a meaning representation language Non-logical vocabulary: open-ended set of names for the
objects, properties and relations (may appear as predicates, nodes, labels on links, labels in slots in frames, etc)
Logical vocabulary: closed set of symbols, operators, quantifiers, links, etc.provide the formal meaning for composing expressions
Each element of non-logical vocabulary must have a denotation in the model. domain of a model: set of objects that are part of the application Capture properties of objects by a set (of domain elements
having the property) Relations denote sets of tuples of elements on the domain
Interpretation: a mapping that maps from the non-logical vocabulary of our meaning representation to the corresponding denotations in the model.
Representational Schemes
We’re going to make use of First Order Predicate Calculus (FOPC) as our representational framework Not because we think it’s perfect All the alternatives turn out to be either too limiting or They turn out to be notational variants
FOPC
Allows for… The analysis of truth conditions
Allows us to answer yes/no questions Supports the use of variables
Allows us to answer questions through the use of variable binding
Supports inference Allows us to answer questions that go beyond what we know
explicitly
FOPC
This choice isn’t completely arbitrary or driven by the needs of practical applications
FOPC reflects the semantics of natural languages because it was designed that way by human beings
In particular…
First-order predicate calculus (FOPC)
Formula AtomicFormula | Formula Connective Formula | Quantifier Variable … Formula | ¬ Formula | (Formula)
AtomicFormula Predicate (Term…) Term Function (Term…) | Constant | Variable Connective ∧ | ⋁ | ⇒ Quantifier ∀ | ∃ Constant A | VegetarianFood | Anarkali Variable x | y | … Predicate Serves | Near | … Function LocationOf | CuisineOf | …
Example
I only have five dollars and I don’t have a lot of time. Have(Speaker,FiveDollars) ∧ ¬ Have(Speaker,LotOfTime) variables:
Have(x,FiveDollars) ∧ ¬ Have(x,LotOfTime) Note: grammar is recursive
Semantics of FOPC
FOPC sentences can be assigned a value of true or false.
Anarkali is near RC. Near (LocationOf (Anarkali), LocationOf (RC))
Inference
Modus ponens:
⇒
Example:
VegetarianRestaurant(Joe’s) x: VegetarianRestaurant(x) ⇒ Serves(x,VegetarianFood)Serves(Joe’s,VegetarianFood)
Uses of modus ponens
Forward chaining: as individual facts are added to the database, all derived inferences are generated
Backward chaining: starts from queries. Example: the Prolog programming language
father(X, Y) :- parent(X, Y), male(X).parent(john, bill).parent(jane, bill).female(jane).male (john).?- father(M, bill).
Variables and quantifiers
A restaurant that serves Mexican food near UM. ∃ x: Restaurant(x)
∧ Serves(x,MexicanFood) ∧ Near(LocationOf(x),LocationOf(UM))
All vegetarian restaurants serve vegetarian food. x: VegetarianRestaurant(x)
⇒ Serves (x,VegetarianFood) If this sentence is true, it is also true for any substitution of x.
However, if the condition is false, the sentence is always true.
Meaning Structure of Language
The semantics of human languages… Display a basic predicate-argument structure Make use of variables Make use of quantifiers Use a partially compositional semantics
Predicate-Argument Structure
Events, actions and relationships can be captured with representations that consist of predicates and arguments to those predicates.
Languages display a division of labor where some words and constituents function as predicates and some as arguments.
Predicate-Argument Structure
Predicates Primarily Verbs, VPs, PPs, Sentences Sometimes Nouns and NPs
Arguments Primarily Nouns, Nominals, NPs, PPs But also everything else; as we’ll see it depends on
the context
Example
Mary gave a list to John. Giving(Mary, John, List) More precisely
Gave conveys a three-argument predicate The first arg is the subject The second is the recipient, which is conveyed by the
NP in the PP The third argument is the thing given, conveyed by
the direct object
Not exactly
The statement The first arg is the subject
can’t be right. Subjects can’t be givers. We mean that the meaning underlying the subject
phrase plays the role of the giver.
Better
Turns out this representation isn’t quite as useful as it could be. Giving(Mary, John, List)
Better would be
),()^,(^),()^,()^(,
ListyIsaxJohnGiveexyGivenxMaryGiverxGivingyx
Predicates
The notion of a predicate just got more complicated… In this example, think of the verb/VP providing a template like the
following
The semantics of the NPs and the PPs in the sentence plug into the slots provided in the template
),()^,()^,()^(,,, xzGiveexyGivenxwGiverxzGivingyxw
Compositional Semantics
Compositional Semantics Syntax-driven methods of assigning semantics to
sentences
Semantic Analysis
Semantic analysis is the process of taking in some linguistic input and assigning a meaning representation to it. There a lot of different ways to do this that make more
or less (or no) use of syntax We’re going to start with the idea that syntax does
matter The compositional rule-to-rule approach
Semantic Processing
We’re going to discuss 2 ways to attack this problem (just as we did with parsing) There’s the theoretically motivated correct and
complete approach… Computational/Compositional Semantics
Create a FOL representation that accounts for all the entities, roles and relations present in a sentence.
And there are practical approaches that have some hope of being useful and successful.
Information extractionDo a superficial analysis that pulls out only the entities, relations and roles that are of interest to the consuming application.
Compositional Analysis
Principle of Compositionality The meaning of a whole is derived from the meanings
of the parts What parts?
The constituents of the syntactic parse of the input What could it mean for a part to have a meaning?
Example
AyCaramba serves meat
),()^,()^( MeateServedAyCarambaeServereServinge
Compositional Analysis
Augmented Rules
We’ll accomplish this by attaching semantic formation rules to our syntactic CFG rules
Abstractly
This should be read as the semantics we attach to A can be computed from some function applied to the semantics of A’s parts.
)}.,....({... 11 semsemfA nn
Example
Easy parts… NP -> PropNoun NP -> MassNoun PropNoun -> AyCaramba MassMoun -> meat
Attachments
{PropNoun.sem}{MassNoun.sem}{AyCaramba}{MEAT}
Example
S -> NP VP VP -> Verb NP Verb -> serves
{VP.sem(NP.sem)} {Verb.sem(NP.sem) ???
),()^,()^( xeServedyeServereServingeyx
Lambda Forms
A simple addition to FOPC Take a FOPC sentence
with variables in it that are to be bound.
Allow those variables to be bound by treating the lambda form as a function with formal arguments
)(xxP
)())((
SallyPSallyxxP
Example
Example
Example
Example
Syntax/Semantics Interface: Two Philosophies
1. Let the syntax do what syntax does well and don’t expect it to know much about meaning
In this approach, the lexical entry’s semantic attachments do all the work
2. Assume the syntax does know something about meaning• Here the grammar gets complicated and the lexicon
simpler (constructional approach)
Example
Mary freebled John the nim.
Where did he get it from?
Who has it?
Why?
Example
Consider the attachments for the VPsVP -> Verb NP NP rule (gave Mary a book)VP -> Verb NP PP (gave a book to Mary)
Assume the meaning representations should be the same for both. Under the lexicon-heavy scheme, the VP attachments are:VP.Sem (NP.Sem, NP.Sem)VP.Sem (NP.Sem, PP.Sem)
Example
Under a syntax-heavy scheme we might want to do something like
VP -> V NP NP V.sem ^ Recip(NP1.sem) ^ Object(NP2.sem) VP -> V NP PP
V.Sem ^ Recip(PP.Sem) ^ Object(NP1.sem) i.e the verb only contributes the predicate, the grammar
“knows” the roles.
Integration
Two basic approaches Integrate semantic analysis into the parser (assign
meaning representations as constituents are completed)
Pipeline… assign meaning representations to complete trees only after they’re completed
Example
From BERP I want to eat someplace near campus
Two parse trees, two meanings
Pros and Cons
If you integrate semantic analysis into the parser as it is running… You can use semantic constraints to cut off parses
that make no sense But you assign meaning representations to
constituents that don’t take part in the correct (most probable) parse
Mismatches
There are unfortunately some annoying mismatches between the syntax of FOPC and the syntax provided by our grammars…
So we’ll accept that we can’t always directly create valid logical forms in a strictly compositional way We’ll get as close as we can and patch things up after
the fact.
Complex Terms
Allow the compositional system to pass around representations like the following as objects with parts:
Complex-Term → <Quantifier var body>
)Restaurant,(xIsax
Example
Our restaurant example winds up looking like
Big improvement…
Meat)Served(e,))RestaurantxxIsaeServereeServing ,(,()(
Conversion
So… complex terms wind up being embedded inside predicates. So pull them out and redistribute the parts in the right way…
P(<quantifier, var, body>)turns intoQuantifier var body connective P(var)
Example
),()Restaurant(
))Restaurant,(,(
xeServerx, Isax
xIsaxeServer
Quantifiers and Connectives
If the quantifier is an existential, then the connective is an ^ (and)
If the quantifier is a universal, then the connective is an -> (implies)
Multiple Complex Terms
Note that the conversion technique pulls the quantifiers out to the front of the logical form…
That leads to ambiguity if there’s more than one complex term in a sentence.
Quantifier Ambiguity
Consider Every restaurant has a menu
That could mean that every restaurant has a menu
Or thatThere’s some uber-menu out there and all restaurants have
that menu
Quantifier Scope Ambiguity
),(),(),()(,)(
MenuyIsayeHadxeHavereyHavingextxRestauran
),(),()(),(),(yeHadxeHavereeHaving
RestaurantxxIsaMenuyyIsa
Ambiguity
This turns out to be a lot like the prepositional phrase attachment problem
The number of possible interpretations goes up exponentially with the number of complex terms in the sentence
The best we can do is to come up with weak methods to prefer one interpretation over another
Non-Compositionality
Unfortunately, there are lots of examples where the meaning (loosely defined) can’t be derived from the meanings of the parts
Idioms, jokes, irony, sarcasm, metaphor, metonymy, indirect requests, etc
English Idioms
Kick the bucket, buy the farm, bite the bullet, run the show, bury the hatchet, etc…
Lots of these… constructions where the meaning of the whole is either Totally unrelated to the meanings of the parts (kick
the bucket) Related in some opaque way (run the show)
The Tip of the Iceberg
Describe this construction1. A fixed phrase with a particular meaning2. A syntactically and lexically flexible phrase with a
particular meaning3. A syntactically and lexically flexible phrase with a
partially compositional meaning4. …
Example
Enron is the tip of the iceberg.NP -> “the tip of the iceberg”
Not so good… attested examples… the tip of Mrs. Ford’s iceberg the tip of a 1000-page iceberg the merest tip of the iceberg
How about That’s just the iceberg’s tip.
Example
What we seem to need is something like NP ->
An initial NP with tip as its head followed bya subsequent PP with of as its head and that has iceberg as the head of its NP
And that allows modifiers like merest, Mrs. Ford, and 1000-page to modify the relevant semantic forms
Quantified Phrases
Consider A restaurant serves meat.
Assume that A restaurant looks like
If we do the normal lambda thing we get
)RestaurantxIsax ,(
)),(,()( MeatServed(e,))RestaurantxxIsaeServereeServing
END
Examples from Russell&Norvig (1)
7.2. p.213
Not all students take both History and Biology. Only one student failed History. Only one student failed both History and Biology. The best history in History was better than the best score in Biology. Every person who dislikes all vegetarians is smart. No person likes a smart vegetarian. There is a woman who likes all men who are vegetarian. There is a barber who shaves all men in town who don't shave themselves. No person likes a professor unless a professor is smart. Politicians can fool some people all of the time or all people some of the time but they
cannot fool all people all of the time.
Categories & Events
Categories: VegetarianRestaurant (Joe’s) – categories are relations and not objects MostPopular(Joe’s,VegetarianRestaurant) – not FOPC! ISA (Joe’s,VegetarianRestaurant) – reification (turn all concepts into
objects) AKO (VegetarianRestaurant,Restaurant)
Events: Reservation (Hearer,Joe’s,Today,8PM,2) Problems:
Determining the correct number of roles Representing facts about the roles associated with an event Ensuring that all the correct inferences can be drawn Ensuring that no incorrect inferences can be drawn
MUC-4 Example
INCIDENT: DATE 30 OCT 89 INCIDENT: LOCATION EL SALVADOR INCIDENT: TYPE ATTACK INCIDENT: STAGE OF EXECUTION ACCOMPLISHED INCIDENT: INSTRUMENT ID INCIDENT: INSTRUMENT TYPEPERP: INCIDENT CATEGORY TERRORIST ACT PERP: INDIVIDUAL ID "TERRORIST" PERP: ORGANIZATION ID "THE FMLN" PERP: ORG. CONFIDENCE REPORTED: "THE FMLN" PHYS TGT: ID PHYS TGT: TYPEPHYS TGT: NUMBERPHYS TGT: FOREIGN NATIONPHYS TGT: EFFECT OF INCIDENTPHYS TGT: TOTAL NUMBERHUM TGT: NAMEHUM TGT: DESCRIPTION "1 CIVILIAN"HUM TGT: TYPE CIVILIAN: "1 CIVILIAN"HUM TGT: NUMBER 1: "1 CIVILIAN"HUM TGT: FOREIGN NATIONHUM TGT: EFFECT OF INCIDENT DEATH: "1 CIVILIAN"HUM TGT: TOTAL NUMBER
On October 30, 1989, one civilian was killed in a reported FMLN attack in El Salvador.
Subcategorization frames
1. I ate2. I ate a turkey sandwich3. I ate a turkey sandwich at my desk4. I ate at my desk5. I ate lunch6. I ate a turkey sandwich for lunch7. I ate a turkey sandwich for lunch at my desk
- no fixed “arity” (problem for FOPC)
One possible solution
1. Eating1 (Speaker)2. Eating2 (Speaker, TurkeySandwich)3. Eating3 (Speaker, TurkeySandwich, Desk)4. Eating4 (Speaker, Desk)5. Eating5 (Speaker, Lunch)6. Eating6 (Speaker, TurkeySandwich, Lunch)7. Eating7 (Speaker, TurkeySandwich, Lunch, Desk)Meaning postulates are used to tie semantics of predicates:
w,x,y,z: Eating7(w,x,y,z) ⇒ Eating6(w,x,y)Scalability issues again!
Another solution
- Say that everything is a special case of Eating7 with some arguments unspecified: ∃w,x,y Eating (Speaker,w,x,y)
- Two problems again:- Too many commitments (e.g., no eating except at meals: lunch, dinner, etc.)- No way to individuate events:
∃w,x Eating (Speaker,w,x,Desk) ∃w,y Eating (Speaker,w,Lunch,y) – cannot combine into ∃w Eating (Speaker,w,Lunch,Desk)
Reification
∃ w: Isa(w,Eating) ∧ Eater(w,Speaker) ∧ Eaten(w,TurkeySandwich) – equivalent to sentence 5.
Reification: No need to specify fixed number of arguments for a given
surface predicate No more roles are postulated than mentioned in the input No need for meaning postulates to specify logical connections
among closely related examples
Representing time
1. I arrived in New York2. I am arriving in New York3. I will arrive in New York
∃ w: Isa(w,Arriving) ∧ Arriver(w,Speaker) ∧ Destination(w,NewYork)
Representing time
∃ i,e,w,t: Isa(w,Arriving) ∧ Arriver(w,Speaker) ∧ Destination(w,NewYork) ∧ IntervalOf(w,i) ∧ EndPoint(I,e) ∧ Precedes (e,Now)
∃ i,e,w,t: Isa(w,Arriving) ∧ Arriver(w,Speaker) ∧ Destination(w,NewYork) ∧ IntervalOf(w,i) ∧ MemberOf(i,Now)
∃ i,e,w,t: Isa(w,Arriving) ∧ Arriver(w,Speaker) ∧ Destination(w,NewYork) ∧ IntervalOf(w,i) ∧ StartPoint(i,s) ∧ Precedes (Now,s)
Representing time
We fly from San Francisco to Boston at 10. Flight 1390 will be at the gate an hour now.
Use of tenses Flight 1902 arrived late. Flight 1902 had arrived late.
“similar” tenses When Mary’s flight departed, I ate lunch When Mary’s flight departed, I had eaten lunch
reference point
Aspect
Stative: I know my departure gate Activity: John is flying
no particular end point Accomplishment: Sally booked her flight
natural end point and result in a particular state Achievement: She found her gate Figuring out statives:
* I am needing the cheapest fare.* I am wanting to go today.* Need the cheapest fare!
Representing beliefs
Want, believe, imagine, know - all introduce hypothetical worlds I believe that Mary ate British food. Reified example:
∃ u,v: Isa(u,Believing) ∧ Isa(v,Eating) ∧ Believer (u,Speaker) ∧ BelievedProp(u,v) ∧ Eater(v,Mary) ∧ Eaten(v,BritishFood)
However this implies also: ∃ u,v: Isa(v,Eating) ∧ Eater(v,Mary) ∧ Eaten(v,BritishFood)
Modal operators: Believing(Speaker,Eating(Mary,BritishFood)) - not FOPC! – predicates
in FOPC hold between objects, not between relations. Believes(Speaker, ∃ v: ISA(v,Eating) ∧ Eater(v,Mary) ∧
Eaten(v,BritishFood))
Modal operators
Beliefs Knowledge Assertions Issues:
If you are interested in baseball, the Red Sox are playing tonight.
Examples from Russell&Norvig (2)
7.3. p.214
One more outburst like that and you'll be in comptempt of court. Annie Hall is on TV tonight if you are interested. Either the Red Sox win or I am out ten dollars. The special this morning is ham and eggs. Maybe I will come to the party and maybe I won't. Well, I like Sandy and I don't like Sandy.