Learning Agents by Prof G. Tecuci

1

Learning Agents LaboratoryComputer Science Department

George Mason University

Prof. Gheorghe Tecuci

2

OverviewOverview

Basic bibliography

Learning strategies

Instructable agents: the Disciple approach

3

Learning strategiesLearning strategies

Inductive learning from examples

Deductive (explanation-based) learning

Introduction

Analogical learning

Abductive learning

Multistrategy learning

4

Machine Learning is the domain of Artificial

Intelligence which is concerned with building

adaptive computer systems that are able to improve

their competence and/or efficiency through learning

from input data or from their own problem solving

experience.

What is Machine LearningWhat is Machine Learning

What does it mean to improve competence?

What does it mean to improve efficiency?

5

Two complementary dimensions for learningTwo complementary dimensions for learning

A system is improving its competence if it learns to solve a broader class of problems, and to make fewer mistakes in problem solving.

A system is improving its efficiency, if it learns to solve the problems from its area of competence faster or by using fewer resources.

Competence

Efficiency

6

The architecture of a learning agentThe architecture of a learning agent

Ontology

Rules/Cases/Methods

Problem SolvingEngine

Learning Agent

User/Environment

Output/

Sensors

Effectors

Input/

Knowledge Base

LearningEngine

Implements learning

methods for extending and refining

the knowledge base to improve agent’s

competence and/or

efficiency in problem solving.

Implements a general problem solving method that uses the knowledge from the knowledge base to interpret the

input and provide an appropriate output.

Data structures that represent the objects from the application domain, general laws governing them, actions that can be performed with them, etc.

7


• Rote learning

• Learning from instruction

• Learning from examples

• Explanation-based learning

• Conceptual clustering

• Quantitative discovery

• Abductive learning

• Learning by analogy

• Instance-based learning

• Case-based learning

• Neural networks

• Genetic algorithms and evolutionary computation

• Reinforcement learning

• Bayesian learning

• Multistrategy learning

A Learning Strategy is a basic form of learning characterized by the employment of a certain type of inference (like deduction, induction or analogy) and a certain type of computational or representational mechanism (like rules, trees, neural networks, etc.).

8

Successful applications of Machine LearningSuccessful applications of Machine Learning

• Learning to recognize spoken words (all of the most successful systems use machine learning);

• Learning to drive an autonomous vehicle on public highway;

• Learning to classify new astronomical structures (by learning regularities in a very large data base of image data);

• Learning to play games;

• Automation of knowledge acquisition from domain experts;

• Instructable agents.

9

Basic ontological elements: instances and conceptsBasic ontological elements: instances and concepts

An instance is a representation of a particular entity from the application domain.

A concept is a representation of a set of instances.

government_of_Britain_1943government_of_US_1943

state_government

instance_of instance_of

“state_government” represents the set of all entities that are governments of states. This set includes “government_of_US_1943” and “government_of_Britain_1943” which are called positive examples.

government_of_US_1943

government_of_Britain_1943

state_government

“instance_of” is the relationship between an instance and the concept to which it belongs.

An entity which is not an instance of a concept is called a negative example of that concept.

10

Concept generalityConcept generality

A concept P is more general than another concept Q if and only if the set of instances represented by P includes the set of instances represented by Q.

state_government

democratic_government

representative_democracy

totalitarian_government

parliamentary_democracy

Example:

“subconcept_of” is the relationship between a concept and a more general concept.

state_government

subconcept_of


11

A generalization hierarchyA generalization hierarchy

feudal_god_king_government

totalitarian_government


theocratic_government

state_government

military_dictatorship

police_state

religious_dictatorship

representative_democracy

parliamentary_democracy

theocratic_democracy

monarchy

governing_body

other_state_government

dictator

deity_figure

chief_and_tribal_council

autocratic_leader

democratic_council_or_board

group_governing_body

other_group_

governing_body

government_of_Italy_1943

government_of_Germany_1943

government_of_US_1943

government_of_Britain_1943

ad_hoc_ governing_body established_ governing_body

other_type_of_governing_body

fascist_state

communist_dictatorshipreligious_

dictatorshipgovernment_

of_USSRy_1943

12


Inductive learning from Examples


Introduction

Analogical learning

Abductive learning


13

Approach:Compare the positive and the negative examples of a concept, in terms of their similarities and differences, and learn the concept as a generalized description of the similarities of the positive examples.

This allows the agent to recognize other entities as being instances of the learned concept.

Illustration

Positive examples of cups: P1 P2 ...

Negative examples of cups: N1 …

A description of the cup concept: has-handle(x), ...

Empirical inductive concept learning from examplesEmpirical inductive concept learning from examples

Given

Learn

14

The learning problemThe learning problem

Given • a language of instances; • a language of generalizations; • a set of positive examples (E1, ..., En) of a concept • a set of negative examples (C1, ... , Cm) of the same concept • a learning bias • other background knowledge

Determine • a concept description which is a generalization of the positive

examples that does not cover any of the negative examples

Purpose of concept learning:Predict if an instance is an example of the learned concept.

15

Generalization and specialization rulesGeneralization and specialization rules

Learning a concept from examples is based on generalization and specialization rules:

-A generalization rule is a rule that transforms an expression into a more general expression;

- A specialization rule is a rule that transforms an expression into a less general expression.

Indicate various generalizations of the following sentence:

Students who have lived in Fairfax for 3 years.

16

Generalization (and specialization) rulesGeneralization (and specialization) rules

Climbing the generalization hierarchy

Dropping condition

Generalizing numbers

Adding alternatives

Turning constants into variables

17

Climbing/descending the generalization hierarchyClimbing/descending the generalization hierarchy

Generalizes an expression by replacing a concept with a more general one.

?O1 is single_state_forcehas_as_governing_body ?O2

?O2 is representative_democracy

generalization specializationrepresentative_democracy democratic_government

The set of single state forces governed by representative democracies

democratic_government representative_democracy

?O1 is single_state_forcehas_as_governing_body ?O2

?O2 is democratic_government

The set of single state forces governed by democracies


representative_democracy parliamentary_democracy

18

Basic idea of version space concept learningBasic idea of version space concept learning

UB+

Initialize the lower bound to the first positive example (LB=E1) and the upper bound (UB) to the most general generalization of E1.

LB

++

UBLBIf the next example is a positive one,

then generalize LB as little as possible to cover it.

_++

UBLB

If the next example is a negative one, then specialize UB as little as possible to uncover it and to remain more general than LB.

_++

UB=LB _

_ ++

…

Repeat the above two steps with the rest of examples until UB=LB.This is the learned concept.

Consider the examples E1, … , E2 in sequence.

19

The candidate elimination algorithmThe candidate elimination algorithm

1. Initialize S to the first positive example and G to its most general generalization

2. Accept a new training instance I• If I is a positive example then

- remove from G all the concepts that do not cover I;- generalize the elements in S as little as possible to

cover I but remain less general than some concept in G;- keep in S the minimally general concepts.

• If I is a negative example then- remove from S all the concepts that cover I;

- specialize the elements in G as little as possible to uncover I and be more general than at least one element from S;

- keep in G the maximally general concepts.

3. Repeat 2 until G=S and they contain a single concept C (this is the learned concept)

20

Illustration of the candidate elimination algorithmIllustration of the candidate elimination algorithm

Language of instances: (shape, size)shape: {ball, brick, cube}size: {large, small}

Learning process:

Input examples:

ball small +

shape size classball large +brick small –cube large –

-(brick, small)

G = {(ball, any-size) (any-shape, large)}

2

-(cube, large)

G = {(ball, any-size)}3

G = {(any-shape, any-size)}

S = {(ball, large)}

+(ball, large)

+(ball, large)

1

1

+(ball, small)

S = {(ball, any-size)}||

4

Language of generalizations: (shape, size)shape: {ball, brick, cube, any-shape}size: {large, small, any-size}

21

General features of the empirical inductive methods General features of the empirical inductive methods

Require many examples.

Do not need much domain knowledge.

Improve the competence of the agent.

The version space method relies on an exhaustive bi-directional search and is computationally intensive. This limits its practical applicability.

Practical empirical inductive learning methods (such as ID3) rely on heuristic search to hypothesize the concept.

22




Introduction

Analogical learning

Abductive learning


23

Explanation-based learning problemExplanation-based learning problem

GivenA training examplecup(o1) color(o1, white), made-of(o1, plastic), light-mat(plastic),

has-handle(o1), has-flat-bottom(o1), up-concave(o1),...Learning goalA specification of the desirable features of the concept to be learned (e.g. the learned concept should have only features from the example)Background knowledgeComplete knowledge that allows proving that the training example represents the concept: cup(x) liftable(x), stable(x), open-vessel(x).

liftable(x) light(x), graspable(x).stable(x) has-flat-bottom(x). …

DetermineA concept definition representing a deductive generalization of the training example that satisfies the learning goal.cup(x) made-of(x, y),light-mat(y),has-handle(x),has-flat-bottom(x),up-concave(x).

Purpose of learningImprove the problem solving efficiency of the agent.

24

Explanation-based learning methodExplanation-based learning method

A example of a cupcup(o1): color(o1, white), made-of(o1, plastic), light-mat(plastic), has-handle(o1), has-flat-bottom(o1), up-concave(o1),...

cup(o1)

stable(o1)liftable(o1)

graspable(o1)light(o1)

light-mat(plastic)

made-of(o1,plastic)

......

has-handle(o1)

cup(x)

stable(x)liftable(x)

graspable(x)light(x)

light-mat(y)made-of(x,y)

......

has-handle(x)

The proof identifies the characteristic features: Proof generalization generalizes them:

• has-handle(o1) is needed to prove cup(o1)

• color(o1,white) is not needed to prove cup(o1)

• made-of(o1, plastic) is needed to prove cup(o1) • made-of(o1, plastic) is generalized to made-of(x, y);• the material needs not be plastic.

1. Construct an explanation that proves that the training example is an example of the concept to be learned.2. Generalize the explanation as much as possible so that the proof still holds, and extract from it a concept definition that satisfies the learning goal.

25

DiscussionDiscussion

How does this learning method improve the efficiency of the problem solving process?

cup(o1)

stable(o1)liftable(o1)

graspable(o1)light(o1)

light-mat(plastic)

made-of(o1,plastic)

......

has-handle(o1)

cup(x)

stable(x)liftable(x)

graspable(x)light(x)

light-mat(y)made-of(x,y)

......

has-handle(x)

The proof identifies the characteristic features Proof generalization generalizes them

Do we need a training example to learn an operational definition of the concept? Why?

The learner does not need a training example. It can simply build proof trees from top-down, starting with an abstract definition of the concept and growing the tree until the leaves are operational features. However, without a training example the learner will learn many operational definitions. The training example focuses the learner on the most typical example.

A cup is recognized by using a single rule rather than building a proof tree.

26

• Needs only one example

• Requires complete knowledge about the concept (which makes this learning strategy impractical).

• Improves agent's efficiency in problem solving

General features of explanation-based learningGeneral features of explanation-based learning

• Shows the importance of explanations in learning

27




Introduction

Analogical learning

Abductive learning


28

Learning by analogyLearning by analogy

Learning by analogy means acquiring new knowledge about an input entity by transferring it from a known similar entity.

sun

planet

yellow

mass

mass

temperature

greater

color

revolves-around

attractsTsun

Tplanet

Msun

Mplanet

causes

temperature

greater

mass

mass

revolves-around

attractsMnucleus

causes ?greater

nucleus

electron

Melectron

sun

planet

yellow

mass

mass

temperature

greater

color

revolves-around

attractsTsun

Tplanet

Msun

Mplanet

causes

temperature

greater

mass

mass

revolves-around

attractsMnucleus

causes ?greater

nucleus

electron

Melectron

The hydrogen atom is like our solar system.

The Sun has a greater mass than the Earth and attracts it, causing the Earth to revolve around the Sun. The nucleus also has a greater mass then the electron and attracts it. Therefore it is plausible that the electron also revolves around the nucleus.

29


Which is the central intuition supporting learning by analogy:If two entities are similar in some respects then they could be similar in other respects as well.

Examples of analogies:

Pressure Drop is like Voltage Drop

A variable in a programming language is like a box.

Provide other examples of analogies.

30

Learning by analogy: the learning problemLearning by analogy: the learning problem

Given:

• A partially known target entity T and a goal concerning it.

• Background knowledge containing known entities.

Find:

• New knowledge about T obtained from a source entity S belonging to the background knowledge.

Partially understood structure of the hydrogen atom under study.

Knowledge from different domains, including astronomy, geography, etc.

In a hydrogen atom the electron revolves around the nucleus, in a similar way in which a planet revolves around the sun.

31

Learning by analogy: the learning methodLearning by analogy: the learning method

• ACCESS: find a known entity that is analogous with the input entity.

• MATCHING: match the two entities and hypothesize knowledge.

• EVALUATION: test the hypotheses.

• LEARNING: store or generalize the new knowledge.

Store that, in a hydrogen atom, the electron revolves around the nucleus. By generalization from the solar system and the hydrogen atom, learn the abstract concept that a central force can cause revolution.

Based on what is known about the hydrogen atom and the solar system identify the solar system as a source for the hydrogen atom.

One may map the nucleus to the sun and the electron to the planet, allowing one to infer that the electron revolves around the nucleus because the nucleus attracts the electron and the mass of the nucleus is greater than the mass of the electron.

A specially designed experiment shows that indeed the electron revolves around the nucleus.

32


How does analogy help?

Why not just study the structure of the hydrogen atom to discover that new knowledge?We anyway need to perform an experiment to test that the electron revolves around the hydrogen atom.

How could we define problem solving by analogy?

How does analogy help to solve new problems?

33

• Requires a huge amount of background knowledge from a wide variety of domains.

• Improves agent's competence through knowledge base refinement.

General features of analogical learningGeneral features of analogical learning

• Is a very powerful reasoning method with incomplete knowledge

• Reduces the solution finding to solution verification.

34




Introduction

Analogical learning

Abductive learning


35

Abductive learningAbductive learning

Finds the hypothesis that explains best an observation (or collection of data) and assumes it as being true.

The learning problem

• Let D be a collection of data• Find all the hypotheses that (causally) explain D• Find the hypothesis H that explains D better than other hypotheses • Assert that H is true

The learning method

Illustration:There is smoke in the East building.Fire causes smoke.Hypothesize that there is a fire in the East building.

36

Abduction (cont.)Abduction (cont.)

Raining causes the streets to be wet.I hypothesize that it has rained on the University Dr.

Provide other examples of abductive reasoning.

Consider the observation: “University Dr. is wet”Use abductive learning.

Which are other potential explanations?

What real world applications of abductive reasoning can you imagine?

37




Introduction

Analogical learning

Abductive learning


38

Multistrategy learningMultistrategy learning

Multistrategy learning is concerned with developing learning agents that synergistically integrate two or more learning strategies in order to solve learning tasks that are beyond the capabilities of the individual learning strategies that are integrated.

39

Examples

Learningfrom examples

Explanation-based learning

Multistrategy

Knowledge

many one several

learning

needed

Effect onagent'sbehavior

improves

competence

improves

efficiency

improvescompetence or/ and efficiency

Type ofinference induction deduction

induction and/ or deduction

Complementariness of learning strategiesComplementariness of learning strategies

complete incompleteknowledge

very littleknowledge

needed

40

OverviewOverview

Basic bibliography

Learning strategies

Instructable agents: the Disciple approach

41

Instructable agents: the Disciple approachInstructable agents: the Disciple approach

Modeling expert’s reasoning

Rule learning

Introduction

Application and evaluation

Multi-agent problem solving and learning

Integrated modeling, learning, solving

42

The knowledge engineer attempts to understand how the subject matter expert reasons and solves problems and then encodes the acquired expertise into the system's knowledge base.

This modeling and representation of expert knowledge is long, painful and inefficient.

How are agents built and why it is hard

KnowledgeEngineer

DomainExpert

Knowledge Base

Inference Engine

Intelligent Agent

ProgrammingDialog

Results

Why?

43

Elaborate a theory and methodology for the mixed-initiative, end-to-end development of knowledge bases

and agents by subject matter experts, with limited assistance from knowledge engineers.

The expert teachesthe agent to perform various tasks in a way that resembles

how the expert would teach a person.

The agent learnsfrom the expert,

building, verifyingand improving itsknowledge base

ApproachMixed-initiative

reasoning between an

expert that has the knowledge

to be formalized and a learning

agent shell that knows how to

formalize it.In

terf

ace

ProblemSolving

Learning

Ontology+ Rules

The Disciple approach: Problem statement

How is this approach addresses the knowledge acquisition bottleneck?

44

MainframeComputers

Software systems developed and used by computer experts

PersonalComputers

Software systems developedby computer experts

and used by persons thatare not computer experts

LearningAgents

Software systems developed and used by persons that are

not computer experts

Inte

rfac

e

ProblemSolving

Learning

Ontology+ Rules

DISCIPLE RKF/COG

Vision on the evolution of computer systems

45

General architecture of Disciple-RKFGeneral architecture of Disciple-RKF

InstancesSolvingRules

Teaching, Learningand Problem Solving

Intelligent User Interface

Autonomous ProblemSolving

Modeling

Knowledge Base Management

OntologyImport

Ontology

InstancesSolvingProblem

Rules

Mixed_Initiative Problem Solving

NaturalLanguage

Generation

OntologyEditors andBrowsers

RuleLearning

RuleRefinement

ScenarioElicitation

TaskLearning

46

Knowledge base: Object ontology + reasoning rules

A hierarchical representation of the types of objects from the COG domain.

<object>

Scenario Strategic_COG_relevant_factor

Established_governing_body

Resource_or_ infrastructure_element

Psychosocial_factor

Geographic_factor

Historic_factor

Military_factor

Political_factor Other_relevant_factor

Civilization_factor

War_scenario

Ad_hoc_governing_body

Other_type_of_governing_body

Demographic_factor

Agent Force_goal

International_factor

Other_political_factor

Controlling_element

Group_governing_body

Governing_body

State_government

Economic_factor

Operation_other_than_war

A hierarchical representation of the types of features of the

objects from the COG domain.

has_as_controlling_leaderDomain: agentRange: person

has_as_religious_leaderDomain: governing_bodyRange: person

has_as_military_leaderDomain: governing_bodyRange: person

has_as_god_kingDomain: governing_bodyRange: person

has_as_monarchDomain: governing_bodyRange: person

has_as_political_leaderDomain: governing_bodyRange: person

has_as_head_of_stateDomain: governing_bodyRange: person

has_as_head_of_governmentDomain: governing_bodyRange: person

has_as_commander_in_chiefDomain: forceRange: person

Object ontology

47

Plausible Lower Bound Condition?O1 is will_of_the_people_of_US_1943?O2 is US_1943

has_as_people ?O6 has_as_governing_body ?O5

?O3 is European_Axis_1943?O4 is Dominance_of_Europe_by_European_Axis?O5 is government_of_US_1943

has_as_will ?O7?O6 is people_of_US_1945 has_as_will ?O1?O7 is will_of_the_government_of_US_1943

reflects ?O1

IF: Test whether the will of the people can make a state accept the strategic goal of an opposing forceThe will of the people is ?O1The state is ?O2The opposing force is ?O3The goal is ?O4

Plausible Upper Bound Condition?O1 is will_of_agent?O2 is force

has_as_people ?O6 has_as_governing_body ?O5

?O3 is (strategic_COG_relevant_factor agent)?O4 is force_goal?O5 is representative_democracy

has_as_will ?O7?O6 is people

has_as_will ?O1?O7 is will_of_agent

reflects ?O1

THEN: Test whether the will of the people, that controls the government, can make a state accept the strategic goal of an opposing force

The will of the people is ?O1The government is ?O5The state is ?O2The opposing force is ?O3The goal is ?O4

Task reduction rule

48

Initial modeling

Ontology specification

Ontology import

Ontology development

Initialobject

ontology

Force

<object>

Scenario

Strategic_goal

Force_goal

Strategic_COG_relevant_factor

Demographic_factor


Psychosocial_factorEconomic_factor

Geographical_factor

Historical_factor

Military_factor

Political_factor

Other_relevant_factor

Operational_goal

Civilization_factor

resource_ or_ infrastructure_element

Problemspecification

Modeling the problem solving process of the subject matter

expert and development of the object ontology of the agent.

Scenarios(elicited)

Generalrules

(learned)has_as_pr im ary_

force_elem ent

ha s asoppo sing

force

Sic ily_1943

“W W II Allied invasiono f Sicily in 1943”

brief_descr iption

Anglo_allies_1943

US_19 43

Brita in_194 3

com ponent_state

compone nt_state

Allied_forces_operation_Husky

type_of_opera tions

has_as_subg roup

US_7 th _Ar my_(Fo rce_ 343)

“combined and joint operations”

has_a s_subgroup

Br_ 8th_A rm y_(Force_545)

has_as_subgroupWe stern_Naval_TFhas_as_subgro up

Eastern_N aval_TFhas_as_subgroup

US_9 th_Air_Fo rcehas_a s_subgroup

Northwest_A frica_A ir_Fo rce

instance_of

instance_of

instance_ofinstance_of

instance_ of

insta nce_of

instance_of

instance_of

instance_of

insta nce_of

insta nce_of

insta nce_of

ha s_as_ industrial_factor

IF the task isIdentify the strategic COG candidates with respect to theindustrial civilization of a state which is a member of a force

The state is ?O2The force is ?O1

THENConclude that an economic factor is a strategic COGcandidate for a state which is a member of a force

The state is ?O2The force is ?O1The economic factor is ?O3

Plausible Upper Bound Condition?O1 IS Force?O2 IS Force

has_as_industrial_factor ?O3?O3 IS Industrial_factor

is_a_major_generator_of ?O4?O4 IS Strategically_essential_goods_o_materiel

Plausible Lower Bound Condition?O1 IS Anglo_allies_1943?O2 IS US_1943

has_as_industrial_factor ?O3?O3 IS Industrial_capacity_of_US_1943

is_a_major_generator_of ?O4?O4 IS War_materiel_and_transports_of_US_1943

explanation?O2 has_as_industrial_factor ?O3?O3 is_a_major_generator_of ?O4?O4 IS strategically_essential_goods_or_materiel














































































































































































































































































































































































































































Force

<object>

Scenario

Strategic_goal

Force_goal

Strategic_COG_relevant_factor

Demographic_factor


Psychosocial_factorEconomic_factor

Geographical_factor

Historical_factor

Military_factor

Political_factor

Other_relevant_factor

Operational_goal

Civilization_factor

resource_ or_ infrastructure_element

Scenario specification

Modeling

Task and rule learning

Task, rule, ontology refinement

Problem solving

Objectontology(refined)

Teaching of the agent by the subject

matter expert.

The developed Disciple approach

49



Rule learning

Introduction




50

Application domain

The center of gravity of an entity (state, alliance, coalition, or group) is the foundation of capability, the hub of all power and movement, upon which everything depends, the point against which all the energies should be directed.

Carl Von Clausewitz, “On War,” 1832.

If a combatant eliminates or influences the enemy’s strategic center of gravity, then the enemy will lose control of its power and resources and will eventually fall to defeat. If the combatant fails to adequately protect his own strategic center of gravity, he invites disaster.

Identification of strategic Center of Gravity candidates in war scenarios

51

Modeling the identification of center of gravity candidatesModeling the identification of center of gravity candidates

Identify and test a strategic COG candidate for the Sicily_1943 scenario

Sicily_1943 is a war scenario

Identify and test a strategic COG candidate for Sicily_1943 which is a war scenario

What kind of scenario is Sicily_1943?

I need to

Therefore I need to

Which is an opposing force in the Sicily_1943 scenario?

Allied_Forces_1943

Identify and test a strategic COG candidate for Allied_Forces_1943

European_Axis_1943

Therefore I need to

Identify and test a strategic COG candidate for European_Axis_1943

Therefore I need to

…

Is Allied_Forces_1943 a single-member force or a multi-member force?

Allied_Forces_1943 is a multi-member force

Identify and test a strategic COG candidate for Allied_Forces_1943 which is a multi-member force

Therefore I need to

…

52

I need to

Test whether the will of the people of US_1943 can make US_1943 accept the strategic goal of European_Axis_1943 which is ‘Dominance of Europe by European Axis’

What is the strategic goal of European_Axis_1943?

‘Dominance of Europe by European Axis’

Therefore I need to

Test the will of the people of US_1943 which is a strategic COG candidate with respect to the people of US_1943

Let us assume that the people of US_1943 would accept ‘Dominance of Europe by European Axis’. Could the people of US_1943 make the government of US_1943 accept ‘Dominance of Europe by European Axis’?

Yes, because US_1943 is a representative democracy and the government of US_1943 reflects the will of the people of US_1943

Therefore I need to

Let us assume that the people of US_1943 would accept ‘Dominance of Europe by European Axis’. Could the people of US_1943 make the military of US_1943 accept ‘Dominance of Europe by European Axis’?

Yes, because US_1943 is a representative democracy and the will of the military of US_1943 reflects the will of the people of US_1943

Therefore conclude that

The will of the people of US_1943 is a strategic COG candidate that cannot be eliminated

Test whether the will of the people of US_1943 that controls the government of US_1943 can make US_1943 accept the strategic goal of European_Axis_1943 which is ‘Dominance of Europe by European Axis’

…

53

Modeling advisor: helping the expert to express his reasoningModeling advisor: helping the expert to express his reasoning

54



Rule learning

Introduction




55

IFIdentify and test a strategic COG candidate corresponding to a member of the ?O1

QuestionWhich is a member of ?O1 ?Answer?O2

THENIdentify and test a strategic COG candidate for ?O2

Natural Language Logic

Rulelearning

IFIdentify and test a strategic COG candidate corresponding to a member of a force

The force is ?O1

THENIdentify and test a strategic COG candidate for a force

The force is ?O2

Plausible Upper Bound Condition ?O1 is multi_member_force

has_as_member ?O2 ?O2 is force

Plausible Lower Bound Condition ?O1 is equal_partners_multi_state_alliance

has_as_member ?O2 ?O2 is single_state_force

explanation?O1 has_as_member ?O2

Identify and test a strategic COG candidate corresponding to a member of the Allied_Forces_1943

US_1943

Identify and test a strategic COG candidate for US_1943

Which is a member of Allied_Forces_1943?

I need to

Therefore I need to

INFORMAL STRUCTURE OF THE RULE

FORMAL STRUCTURE OF THE RULE

56

1. Formalize the tasks1. Formalize the tasks


I need to

Therefore I need to


Identify and test a strategic COG candidate for a force

The force is US_1943

I need to

Therefore I need to

Identify and test a strategic COG candidate corresponding to a member of a force

The force is Allied_Forces_1943

57

Task formalization interfaceTask formalization interface

58

2. Find an explanation of why the example is correct 2. Find an explanation of why the example is correct

US_1943has_as_member

Allied_Forces_1943

The explanation is the best possible approximation of the question and the answer, in the object ontology.

US_1943



I need to

Therefore I need to


59

Explanation generation and selection interfaceExplanation generation and selection interface

60

3. Generate the plausible version space rule3. Generate the plausible version space rule

Condition?O1 is Allied_Forces_1943

has_as_member ?O2?O2 is US_1943

Most general generalization


The force is ?O1


The force is ?O2






Most specific generalization

US_1943has_as_member

Allied_Forces_1943

Rewriteas

Which is the justification for these generalizations?

61

Analogical reasoningAnalogical reasoning

similar exampleexplains?

similar

Identify and test a strategic COG candidate for a force The force is Germany_1943

I need to

Therefore I need to

Identify and test a strategic COG candidate corresponding to a member of a force The force is European_Axis_1943

initial example

explanation

explains

Identify and test a strategic COG candidate for a force The force is US_1943

I need to

Therefore I need to

Identify and test a strategic COG candidate corresponding to a member of a force The force is Allied_Forces_1943

US_1943has_as_memberAllied_Forces_1943

similar explanation

less general than

Analogycriterion

less general than

?O2has_as_member?O1

forcemulti_member_force


Germany_1943has_as_memberEuropean_Axis_1943

similar

62

Generalization by analogyGeneralization by analogy

Any value of ?O1 should be an instance of: DOMAIN(has_as_member) RANGE(The force is) =

multi_member_force force = multi_member_forceAny value of ?O2 should be an instance of: RANGE(has_as_member) RANGE(The force is) = force force = force

Knowledge-base constraints on the generalization:

initial example

explains

Identify and test a strategic COG candidate for a force The force is US_1943

I need to

Therefore I need to

Identify and test a strategic COG candidate corresponding to a member of a force The force is Allied_Forces_1943

US_1943has_as_memberAllied_Forces_1943

generalization explains

?O2has_as_member?O1

forcemulti_member_force


Identify and test a strategic COG candidate for a force The force is ?O2

I need to

Therefore I need to

Identify and test a strategic COG candidate corresponding to a member of a force The force is ?O1

63



Rule learning

Introduction




64

Control of modeling, learning and solvingControl of modeling, learning and solving

Input Task

Generated Reduction

Mixed-Initiative Problem Solving

Ontology + Rules

Reject ReductionAccept ReductionNew Reduction

Rule Refinement

Task RefinementRule Refinement

Modeling

Formalization

Learning

Solution

65

US_1943



I need to

Therefore I need to


Provides an example

1

Rule_15Learns

2

Rule_15

?

Applies

Germany_1943

Identify and test a strategic COG candidate for Germany_1943

Which is a member of European_Axis_1943?

Therefore I need to

3

Accepts the example

4Rule_15Refines

5I need to

Identify and test a strategic COG candidate corresponding to a member of the European_Axis_1943

…

66

Rule refinement with a positive exampleRule refinement with a positive example

Condition satisfiedby positive example

?O1 is European_Axis_1943 has_as_member ?O3

?O2 is Germany_1943le

ss g

ener

al t

han

Positive example thatsatisfies the upper bound

explanationEuropean_Axis_1943 has_as_member Germany_1943


The force is ?O1


The force is ?O2





explanation?O1 has_as_member ?O2 Identify and test a strategic COG

candidate for Germany_1943

I need to

Therefore I need to

Identify and test a strategic COG candidate corresponding to a member of the European_Axis_1943

67

Condition satisfied by the positive example

?O1 is European_Axis_1943 has_as_member ?O2

?O2 is Germany_1943

Plausible Upper Bound Condition

?O1 is multi_member_force has_as_member ?O2

?O2 is force

Plausible Lower Bound Condition (from rule)

?O1 is equal_partners_multi_state_alliance has_as_member ?O2

?O2 is single_state_force

Minimal generalization of the plausible lower boundMinimal generalization of the plausible lower bound

minimal generalization

less general than (or at most as general as)

New Plausible Lower Bound Condition?O1 is multi_state_alliance

has_as_member ?O2

?O2 is single_state_force

68

single_state_force single_group_force multi_state_force multi_group_force

multi_state_alliance multi_state_coalition

equal_partners_multi_state_alliance

dominant_partner_multi_state_alliance

equal_partners_multi_state_coalition

dominant_partner_multi_state_coalition

composition_of_forces

multi_member_forcesingle_member_force

Forces hierarchyForces hierarchy

Allied_Forces_1943

European_Axis_1943

force

…

Germany_1943US_1943

multi_state_alliance is the minimal generalization of equals_partners_multi_state_alliance that covers European_Axis_1943

69

Refined ruleRefined rule

less

gen

eral

th

an


The force is ?O1


The force is ?O2







The force is ?O1


The force is ?O2



Plausible Lower Bound Condition ?O1 is multi_state_alliance



70

Which are some learning strategies used by Disciple?Which are some learning strategies used by Disciple?

Example ofproblem solving

episode

PVS Rule

analogy

exactcondition

PLB

PUB

Knowledge Base

Incompleteexplanation

Rule learning

Knowledge Base

Failureexplanation

PVSRule

Example solutionsgenerated by the agent

Incorrectexample

Correctexample

Rule refinement

71



Rule learning

Introduction




72

Students use Disciple as an intelligent assistant that supports them to develop a Center of Gravity analysis report for a war scenario.

319jw Case Studies in Center of Gravity Analysis (COG), Term II and Term III

Students act as subject matter experts thatteach personal Disciple agents their own reasoning in Center of Gravity analysis.

589 Military Applications of Artificial Intelligence (MAAI) ,Term III

Students develop

inputscenarios

Studentsdevelopagents

Use of Disciple in Army War College courses

Inte

rfac

e

ProblemSolving

Learning

Ontology+ Rules

DISCIPLE RKF/COG

73

0

1

2

3

4

5

6

7

8

9

10

11

StronglyDisagree

Disagree Neutral A gree StronglyA gree

0

1

2

3

4

5

6

7

8

9

10

11

StronglyDisagree


0

1

2

3

4

5

6

7

8

9

10

11

StronglyDisagree


0

1

2

3

4

5

6

7

8

9

10

11

StronglyDisagree


The use of Disciple is an assignment

that is well suited to the course's

learning objectives

The use of Disciple wasa useful learning

experience

Disciple helped me to learn to perform

a strategic center of gravity analysis of a

scenario

Disciple should be used in

future versions of this course

COG Winter 2002: Expert assessments

74

I think that a subject matter expert can use Disciple to build an agent, with limited assistance from a knowledge engineer

589 MAAI Spring 2002: SME assessment589 MAAI Spring 2002: SME assessment

0123456789

StronglyDisagree

Disagree Neutral Agree StronglyAgree

75



Rule learning

Introduction




76

Modeling, problem solving, and learning as collaborative agentsModeling, problem solving, and learning as collaborative agents

Reasoning tree similar with the current model

Learning

Modeling

Problem Solving

Mixed

Initiative

New situations that need to be modeled

New positive and negative examples for rule refinement

Analysis of the instantiations for a rule

Partially learned rules with informal and formal descriptions

Rules similar with the current example

Informal description of the reasoning

New correct examples for learning

ExampleAnalyzer

ExampleComposer

Rule Analyzer

Explanation Generator

Rule Analogy Engine

Solution Analyzer

Reasoning Tree Analyzer

Mixed-Initiative PS

77

Question-answering based task reduction

S1

S11a

S1n

S11b1 S11bm

T11bmT11b1

T1nT11a

…

…

T1

Q1

S11bT11b

A1n S11A11

……

A11b1 A11bm

S11bQ11b

Let T1 be the problem solving task to be performed.

Finding a solution is an iterative process where, at each step, we consider some relevant information that leads us to reduce the current task to a simpler task or to several simpler tasks.

The question Q associated with the current task identifies the type of information to be considered.

The answer A identifies that piece of information and leads us to the reduction of the current task.

A complex problem solving task is performed by:

• successively reducing it to simpler tasks;

• finding the solutionsof the simplest tasks;

• successively composing these solutions until the solution to the initial task is obtained.

78

Assess whether the political environment would support a

training base for terrorist operations at Location-A

Assess whether Location-A is a training base for terrorist operations

What type of factors should be considered to assess the presence of a terrorist training base?

Assess whether there is a flow of suspected

terrorists in the region of Location-A

Assess whether there are other suspected bases for terrorist operations in the

region of Location-A

Assess whether the physical structures at Location-A support the existence of a training base

for terrorist operations

Political environment, physical structures, flow of suspected terrorists, weapons and weapons technology, other suspected bases in the region, and terrorist sympathetic population

Assess whether there are weapons and weapons technology at Location-A that suggest the presence of a training base

for terrorist operations

Ta

sk R

ed

uc

tion

Conclude that there are indicators of the presence of weapons (plastic explosives, firearms) at Location-A but …

Conclude that there are strong indicators that

Location-A is a training base for terrorist operations

Assess whether there is terrorist sympathetic

population in the region of Location-A

So

luti

on

Syn

thes

is

Assess whether there are explosive experts

in the vicinity of Location-A

What things are indicators of the presence of plastic explosives?

Assess whether there is evidence of presence of detonation equipment at

Location-A

Assess whether there is evidence of presence of secure storage facilities

at Location-A

Location-A contains a building with a

continuous security guard …

Conclude that there are fairly strong indicators of the presence of plastic explosives at Location-A

Assess whether there are indicators of the presence of plastic explosives at Location-A

Assess whether there is evidence of past use of plastic explosives at

location-A

Person-E, a known explosive expert, has been seen in the vicinity of

Location-A by a credible source, on several occasions

These are fairly strong indicators

What kind of indicators are these for the presence of plastic

explosives at Location_A?

Person-X has reported a plastic explosion on

10 August 2001 at Location-A

Historical evidence of past explosions, presence of explosive experts in the area, presence of detonation equipment, and presence of secure storage facilities

There is a credible report that detonation equipment was brought into the country of Location-A by

truck shipment in July 2001

Intelligence Analysis

79

EXPERT

PERSONALIZED LEARNING ASSISTANT

Local Shared KB

PERSONALIZEDLEARNINGASSISTANT

Expert’s Model

External-Expertise

Agent

Information-Interaction

Agent

AwarenessAgent

SupportAgent

SupportAgent

SupportAgent

Ontology

Rules

Global Grid Computing and Communications Infrastructure

JBI Platform – minimal set of standards and services

++ + +

KB

KB KB

KB

KB

Global KB

EXPERT

A possible multi-agent architecture for Disciple

80

BibliographyBibliography

Mitchell T.M., Machine Learning, McGraw Hill, 1997.

Shavlik J.W. and Dietterich T. (Eds.), Readings in Machine Learning, Morgan Kaufmann, 1990.

Buchanan B., Wilkins D. (Eds.), Readings in Knowledge Acquisition and Learning: Automating the Construction and the Improvement of Programs, Morgan Kaufmann, 1992.

Langley P., Elements of Machine Learning, Morgan Kaufmann, 1996.

Michalski R.S., Carbonell J.G., Mitchell T.M. (Eds), Machine Learning: An Artificial Intelligence Approach, Morgan Kaufmann, 1983 (Vol. 1), 1986 (Vol. 2).

Kodratoff Y. and Michalski R.S. (Eds.) Machine Learning: An Artificial Intelligence Approach (Vol. 3), Morgan Kaufmann Publishers, Inc., 1990.

Michalski R.S. and Tecuci G. (Eds.), Machine Learning: A Multistrategy Approach (Vol. 4), Morgan Kaufmann Publishers, San Mateo, CA, 1994.

Tecuci G. and Kodratoff Y. (Eds.), Machine Learning and Knowledge Acquisition: Integrated Approaches, Academic Press, 1995.

Tecuci G., Building Intelligent Agents: An Apprenticeship Multistrategy Learning Theory, Methodology, Tool and Case Studies, Academic Press, 1998.

Documents

Learning Agents by Prof G. Tecuci