Words and Meaning - Lunds tekniska högskola · Web viewThe Naïve Bayes model is based on the bag-of-word approach: it assumes that words occur independently of each other. We replace

Language Processing and Computational Linguistics

EDA171/DATN06 – Lecture 13Lexical Semantics

Richard [email protected]

http://www.cs.lth.se/~richard

© Pierre Nugues, Lecture 13, September 30, 2008 1

mailto:[email protected]

Today’s Lecture

Lexical Semantics: how do we represent and process the meaning of words? The structure of dictionaries. Ontologies / semantic networks. Word senses. Sense ambiguity. Sense disambiguation. Putting pieces together: valence, selectional

restrictions, …


Words and Meaning

Referred to as lexical semantics: Classes of words: If it is hot, can it be cold? Definition What is a meal? What is table? Reasoning: The meal is on the table. Is it cold?


Categories of Words

Expressions, which are in no way composite, signify substance, quantity, quality, relation, place, time, position, state, action, or affection. To sketch my meaning roughly, examples of substance are ‘man’ or ‘the horse’, of quantity, such terms as ‘two cubits long’ or ‘three cubits long’, of quality, such attributes as ‘white’, ‘grammatical’. ‘Double’, ‘half’, ‘greater’, fall under the category of relation; ‘in the market place’, ‘in the Lyceum’, under that of place; ‘yesterday’, ‘last year’, under that of time. ‘Lying’, ‘sitting’, are terms indicating position, ‘shod’, ‘armed’, state; ‘to lance’, ‘to cauterize’, action; ‘to be lanced’, ‘to be cauterized’, affection.

Aristotle, Categories, IV. (trans. E. M. Edghill)© Pierre Nugues, Lecture 13, September 30, 2008 4

Representation of Categories

substance quality state relation action position place time quantity affection

expressions


Definitions

Short texts describing a word: A genus or superclass using a hypernym. Specific attributes to differentiate it from other

members of the superclass. This part of the definition is called the differentia specifica.

bank (1.1): a land sloping up along each side of a canal or a river.hedgehog: a small animal with stiff spines covering its back.waiter: a person employed to serve customers at their table in a restaurant, etc .


Relations between Words in a Dictionary

Synonymy/Antonymy Hyponyms/Hypernyms is_a(tree, plant), life form, entity Meronyms/Holonyms part_of(leg, table) …


Semantic Networks – Ontologies

food

animates

human beings furnitureanimals

substance

possess

eat

carnivoresinsectivores

meatmammals

eat


An Example: WordNet

Based on the concept of the synset.Nouns Hyponyms/hypernyms

Synonyms/antonymsMeronyms/holonyms

Adjectives Synonyms/antonymsRelational fraternal --> brother

Verbs Semantic domains (body function, change, communication, perception, contact, motion, creation, possession, competition, emotion, cognition, social interaction, weather)Synonymy, Antonymy: (rise/fall, ascent/descent, be born/die)“Entailment”: succeed/try, snore/sleep


Lexical Database

%% is_a(?Word, ?Hypernym)is_a(hedgehog, insectivore).is_a(cat, feline).is_a(feline, carnivore).is_a(insectivore, mammal).is_a(carnivore, mammal).is_a(mammal, animal).is_a(animal, animate_being).

hypernym(X, Y) :- is_a(X, Y).hypernym(X, Y) :- is_a(X, Z), hypernym(Z, Y).


Semantics and Reasoning

The caterpillar ate the hedgehog.Representation:∃(X, Y), caterpillar(X) & hedgehog(Y) & ate(X, Y).

Reasoning (inference):It is untrue because the query:?- predator(X, hedgehog)X = foxes, eagles, car drivers, …but no caterpillar.


Word ambiguity: Homonymy and Polysemy

Words are ambiguous: A single form may have more than one entry (homonymy) and sense (polysemy).The Oxford Advanced Learner’s Dictionary (OALD) lists five entries for bank: Raised ground Organization Turn.and five senses for the first entry.The distinction between homonymy and polysemy is sometimes problematic. WordNet does not distinguish.Polysemy is productive! “bank of English”


Sense Distinctions Vary Across Languages

Granularity: Boundaries:Swedish English French Welshsätta2 gwyrdd

ställa1 put1 vert

lägga1 bleuglas

grisllwyd

brun


Sense Tagging Using the OALD

The patron ordered a mealWords Definitions Sensepatron Correct sense: A customer of a shop, restaurant, theater 1.2

Alternate sense: A person who gives money or support to a person, an organization, a cause or an activity

1.1

ordered Correct sense: To request somebody to bring food, drink, etc in a hotel, restaurant etc.

2.3

Alternate senses: To give an order to somebodyTo request somebody to supply or make goods, etc.To put something in order

2.12.22.4

Meal Correct sense: The food eaten on such occasionAlternate sense: An occasion where food is eaten

1.21.1


Identifying Senses Automatically

The problem of determining sense is called word sense disambiguation.This is one of the most difficult tasks in NLP – the best systems are only slightly better than the baseline.Two broad approaches: Statistical methods based on co-occurrence statistics Dictionary-based methods

Hybrids are common.Both approaches rely crucially on the context to disambiguate.


Bayesian Sense Disambiguation

We analyze the interaction between bank and market finance in a model where bankers gather information through monitoring…Statistical techniques optimize a sequence of semantic tags.The context

€

C is defined as

€

w−m ,w−m +1,...,w−1,w,w1,...,wm−1,wm .The optimal sense

€

ˆ s = argmaxsi ,1≤ i≤n

P(si | C).Using Bayes’ theorem, we have:

)|,,...,,,...,,()(maxarg

)|()(maxargˆ

11111,

1,

immmminis

iinis

swwwwwwPsP

sCPsPs

i

i

−−+−−≤≤

≤≤

=

=


Naïve Bayes

The Naïve Bayes model is based on the bag-of-word approach: it assumes that words occur independently of each other. We replace )|,,...,,,...,,( 1111 immmm swwwwwwP −−+−− with the product of probabilities:

∏≠−=≤≤

=m

jmjiji

nisswPsPs

i 0,1,)|()(maxargˆ

To train the model, we can use SemCor, which is a sense-annotated corpus for English.In addition to supervised methods like Naïve Bayes, semisupervised and unsupervised algorithms exist.


Using Dictionaries (Lesk and derived methods)

We analyze the interaction between bank and market finance in a model where bankers gather information through monitoring and screeningMaximally overlapping definitions:Bank: Bank1: The land sloping up along each side of a river or a canal; he

ground near a river Bank2: An organization or a place that provides a financial service.

Customers keep their money in the bank safely and it is paid out…Finance: Finance: The money used or needed to support an activity, project,

etc; the management of money


Valence Patterns

Dictionaries store information about how words combine with other words to form larger structures. This information is called valence (cf. valence in chemistry)

OALD, tell1:~ sb (sth) / ~ sth to sb

I told him a lie I told a lie to him


Syntactic Side: Verb Construction Models

English French Germandepend + on + object noun group

dépendre + de + object noun group

hängen + von + dative noun group + ab

I like + verb-ing (gerund)

Ça me plaît de + infinitive

es gefällt mir + zu + infinitive

require + verb-ing (gerund)

demander + de + infinitive

verlangen + accusative noun group


Semantic Side: Selectional Restrictions

Three kinds of wanting:1. Wanting something to happen,2. Wanting an object,3. Wanting a person.and (2.) will be mapped on:word(category: verb, agent: persons, object: objects) --> [want].Properties of word mean: adjective, qualify only persons, and express badness:word(category: adjective, applyTo: persons, expresses: badness)--> [mean].


Case Grammar

An idea from Indian grammarians (~500 BC): Define a set of semantic cases (or semantic roles):•An Agent – Instigator of the action (typically animate)•An Instrument – Cause of the event or object in causing the event

(typically animate)•An Object – Entity that is acted upon or that changes, the most

general case.•A Dative – Entity affected by the action (typically animate)•A Locative – Place of the event•…Used by linguists to study the relations between valence patternsUsed in NLP to build flexible template-filling applications


Case Grammar: An Example

open(O, {A}, {I})

The door opened O = doorJohn opened the door O = door and A = JohnThe wind opened the door O = door and I = windJohn opened the door with a chisel

O = door, A = John, and I = chisel


Parsing with Cases

The waiter brought the meal to the patronIdentify the verb bring and apply constraints:

Bring

Case Value

Agentive Animate (Obligatory) The waiter

Object (Obligatory) the mealDative Animate (Optional) the

patronTime (Obligatory) past


FrameNet

In 1968, Fillmore wrote an oft-cited paper on case grammars.Later, he started the FrameNet project.In FrameNet, Fillmore no longer uses universal cases but a set of frames – semantic templates – where each frame is specific to a class of words.Framenet is a medium-sized lexical database that maps English words to frames.Examples of frames: Revenge, Statement, Impact, …


The Revenge Frame

15 lexical units (dictionary entries):avenge.v, avenger.n, get back (at).v, get_even.v, retaliate.v, retaliation.n, retribution.n, retributive.a, retributory.a, revenge.n, revenge.v, revengeful.a, revenger.n, vengeance.n, vengeful.a, and vindictive.a.Examples of frame elements (template slots)Avenger, Punishment, Offender, Injury, and Injured_party.The word in a sentence that is tied to a frame is called the target.


Annotation

1. [<Avenger> His brothers] avenged [<Injured_party> him].2. [<Punishment>With this], [<Avenger> El Cid] at once avenged

[<Injury> the death of his son].3. [<Avenger> Hook] tries to avenge [<Injured_party> himself]

[<Offender> on Peter Pan] [<Punishment> by becoming a second and better father].


Automatic Frame-semantic Analysis

Given a syntax tree and a target word Find the arguments, Determine the template slot of each argument

to l dhim a lie I

SBJ IO B J

OBJRO O T


Classification of Semantic Arguments

The task of classifying semantic arguments can be modeled as a statistical classification problem.

What features are useful for this task? Examples: Grammatical function: subject, object, … Voice: “I told a lie” / “I was told a lie” Semantic classes: “I told him” / “the note told him” Semantic class usually not available: use word instead


Example

Given a dependency tree:

to l dhim a lie I

SBJ IO B J

OBJRO O T

We select the three dependents of told:I: GF = Subject voice = Active word = Ihim: GF = Indirect object voice = Active word = himlie: GF = Direct object voice = Active word = lie


When Semantics Is Too Difficult: “Semantic Grammar”

sentence --> npInsectivores, ingest, npCrawlingInsects.npInsectivores --> det, insectivores.npCrawlingInsects --> det, crawlingInsects.insectivores --> [mole].insectivores --> [hedgehog].ingest --> [devours].ingest --> [eats].crawlingInsects --> [worms].crawlingInsects --> [caterpillars].det --> [the].


An Application: EVAR

EVAR was a German project to provide information on trains.

noun

concrete abstract

thing location animate worth classifying time

transport human beast


EVAR’s Valence Dictionary

1. fahren1.1(”The train is going from Hamburg to Munich”) Instrument: noun group (nominative), Transport, obligatory Source: prepositional group (Origin), Location, optional Goal: prepositional group (Direction), Location, optional

2. fahren1.2(”I am going by train from Hamburg to Munich”) Agent: noun group (nominative), Animate, obligatory Instrument: prepositional group (prep = mit), Transport,

optional Source: prepositional group (Origin), Location, optional Goal: prepositional group (Direction), Location, optional

3. Abfahrt1.1(”the departure of the train at Hamburg for Munich”) Object: noun group (genitive), Transport, optional


Another Application: Traffic Accidents

[<Location>På landsvägen mellan Lund och Staffanstorp] krockade [<Time>i morse] [<Actor>en motorcyklist] [<Manner> våldsamt] [<Victim>med ett träd].

… efter en [<Manner>oförsiktig] omkörning …


Documents

Words and Meaning - Lunds tekniska högskola · Web viewThe Naïve Bayes model is based on the bag-of-word approach: it assumes that words occur independently of each other. We replace