Some Thoughts on Machine Understanding

Peter Clark

Knowledge Systems

Boeing Engineering and Information Technology

On Machine Understanding• Understanding = creating a situation-specific model

(SSM), coherent with data & background knowledge– Data suggests model fragments which may be appropriate– Models suggest ways of interpreting data

Garbled graphof relationships Coherent Model

(situation-specific)

?

?

On Machine Understanding

Garbled graphof relationships Coherent Model

(situation-specific)

?

?

• Core theories of the world• Ton of common-sense/ episodic/experiential knowledge (“the way the world is”)

Assembly of pieces, assessment of coherence,inference

• Only a tiny part of the target model• Contains errors and ambiguity• Not even a subset of the target model

What are some ingredients?

1. Elaboration (“scene building”)

2. Representing possibilities

3. Coherence assessment (“matching”?)

4. Viewpoints/context

5. Knowledge acquisition

1. Elaboration:The Parachute Sentences

“Parachutes slow down a person falling through the air. This means that he or she can land safely when jumping out of a plane. When open, a parachute creates lots of drag as air pushes against its underside. This slows its fall.”

The Parachute Sentences

“Parachutes slow down a person falling through the air. This means that he or she can land safely when jumping out of a plane. When open, a parachute creates lots of drag as air pushes against its underside. This slows its fall.”

1. Elaboration (cont)

• A vivid picture comes to mind– John: adult male, out in woods– holding an axe (or chain saw?)– Tree is ~30ft high pine tree

• or: a modification of that time I sawed a Christmas tree• or: that documentary on logging in Canada

• 8:1 ratio of prior to explicit knowledge (Graesser, ’81)• Episodic/experiential knowledge plays a key role• Also core knowledge plays a key role• Not a deductive process!

“John chopped down the tree.”

1. Elaboration: Using WordNet

“The kid hit the ball hard.”

• Augment “semantic structure” with definitional “knowledge”.

1. Elaboration: Using WordNet

“The kid hit the ball hard.”

• Augment “semantic structure” with definitional “knowledge”.

1. Elaboration: Another example

• satellite: orbit around earth; receive/send radio messages

• navigation: information about location

• system: assembly of artifacts which together perform a task

• people: often want to know where they are

• (after more sentences): entire model on how GPS systems work.

“The Global Positioning System is a satellite navigation system designed to provide instantaneous position, velocity and time information almost anywhere on the globe.”

2. Representing Possibilities• Went to encode a space of possibilities

– not what the model is, but constraints on what the actual models might be

– enable actual models to be built and assessed

“Most eucaryotic genes have their coding sequences interrupted by noncoding sequences, called introns. The scattered pieces of coding sequence, called exons, are usually shorter than the introns, and the coding portion of a gene is often only a small fraction of the total length of the gene. Most introns range in length from about 80 nucleotides to 10,000 nucleotides, although even longer introns exist.”

p220, Alberts 1998.

where?

2. Representing Possibilities

Spaces of possible models

Possible (consistent)

Impossible (inconsistent)

More likely/preferred

Less likely/preferred

Got Want

2. Representing Possibilities• Are a few (feeble) methods in KM for this:

– type restrictions(every Person has (spouse ((must-be-a Person)))))

– (sometimes <x>) • (every Car has (parts ((sometimes (a Spare-Wheel)))

– Cardinality constraints• (a Group with (min-cardinality (…)) (max-cardinality (…)))

– Range of values• “size is between X and Y”

• Still largely lacking in how to represent and reason with vague knowledge

3. What Makes a Representation (Model) Coherent?

• We don’t just blindly accept new knowledge:– Minsky: We proactively ask a set of pertinent questions about a

scene, e.g., what is X for? What are the goals? etc.

• What makes a representation coherent?– Simple consistency (“The man fired the gnu.”)

– Purposefulness (for artifacts):

• “The engine contains a thrust reverser.”

• vs. “The engine contains an elephant.”

• vs. “The engine contains a book.”

• “Knowledge entry” is a serious misnomer!– Really talking about Knowledge Integration

Transponder parts: antenna purpose: receive, transmit

“TRANSPONDER: A combination receiving and transmitting antenna on a communications satellite. TRANSPONDER: A combination receiver, frequency converter, and transmitter package, physically part of a communications satellite.

3. What Makes a Representation (Model) Coherent?

Transponder parts: receiver frequency converter transmitter part-of: communications satellite

Relay/Mediate

3. A Catalog of Coherence Criterea

1. Volitional actions:– Agents must be capable of an action

• legally, skill, fiscally, anatomically

– Action serves a broader purpose/goal– Need equipment/resources/instruments, instruments must

be adequate

2. Non-volitional actions;– There is a cause (inc. randomness)– Spatial:

• statics: objects must be close• dynamics: objects can move in the required way

– Temporal: objects exist at the same time

3. A Catalog of Coherence Criterea (cont)

3. Objects:– physically possible

• parts connected together at appropriate places• materials are appropriate• suspension/tension etc., gravity

– physically normal/expected/standard• need to know normal shapes, sizes, etc.

4. Artifacts:– Purposefulness:

• all parts play some role wrt. one of its intended functions (or subtasks thereof). Expect design to be optimized.

– Could treat biological objects as “artifacts”

3. Coherence and KM

• KM unable to tolerate incoherence:– Current: “Error! Switching on the debugger…”

– Desired: “This representation is generally ok, except this bit looks weird, and that bit conflicts with this bit.”

• Problem compounded by long inference chains– (cf. Cyc: don’t think too hard )

• How could we change KM to be more tolerant?

4. Viewpoints and Context

• Latter seems right, but:– can a big KB really be partitioned like this? (everything is

connected!)• Models may vary by:

– ignoring detail– making different approximations– using different ontologies

Reason overGiant KB

vs.

Problem-specific KB,contains selected units

Componenttheories/

ontologies(?)

4. Viewpoints and Context• e.g., DNA = sequence of different region types:

– intron-exon-intron-exon…

– promotor-gene-terminator

– nucleotide pair-nuc pair-nuc pair…

• Makes a difference:– Given: “The polymerase attaches to the promotor, and then moves down

the strand.”

– then answer: “Where will the polymerase be?”• nucleotide? gene? intron?

• The point: It’s not simply a matter of having all viewpoints coexisting

• Another example: “A satellite sends signals/messages/position information.”

5. Knowledge Acquisition• Where do the core theories come from?

– Hand engineered?

• Where does all the “mundane” knowledge come from?– Schubert-style?

– Dictionary/glossary definitions?

5. Knowledge Acquisition:Can all this be Bootstrapped?

Text

List of compoundnouns and verbs

(entities and actions)

DomainOntology

Jungle ofparse trees/

semantic graphs

Collection ofcoherent scenes

KB

Scenelibrary