Learning Agents Center George Mason University Symposium on Reasoning and Learning in Cognitive Systems Stanford, CA, 20-21 May 2004 Gheorghe Tecuci with

Learning Agents CenterGeorge Mason University

Symposium on Reasoning and Learning in Cognitive SystemsStanford, CA, 20-21 May 2004

Gheorghe Tecuciwith Mihai Boicu, Dorin Marcu, Bogdan Stanescu,

Cristina Boicu, Marcel Barbulescu

Problem Solving Method: Task Reduction

Learnable Knowledge Representation: Plausible Version Spaces

Multistrategy Learning during Problem Solving

Overview

Agent Development Experiments

Teaching and Learning Demo

Acknowledgements

Research Problem, Approach, and Application

Future Directions: Life-long Continuous Learning

The expert teachesthe agent to perform various tasks in a way that resembles

how the expert would teach a person.

The agent learnsfrom the expert,

building, verifyingand improving itsknowledge base

1. Mixed-initiative problem solving

2. Teaching and learning

3. Multistrategy learning In

terf

ace

ProblemSolving

Learning

Ontology+ Rules

Research Problem and Approach

Research Problem: Elaborate a theory, methodology and family of tools for the development of knowledge-base agents by subject matter experts, with limited assistance from knowledge engineers.

Approach: Develop a learning agent that can be taught directly by a subject matter expert while solving problems in cooperation.

The center of gravity of an entity (state, alliance, coalition, or group) is the foundation of capability, the hub of all power and movement, upon which everything depends, the point against which all the energies should be directed.

Carl Von Clausewitz, On War, 1832.

If a combatant eliminates or influences the enemy’s strategic center of gravity, then the enemy will lose control of its power and resources and will eventually fall to defeat. If the combatant fails to adequately protect his own strategic center of gravity, he invites disaster. Giles and Galvin, USAWC 1996.

Sample Domain: Center of Gravity Analysis

The center of gravity of an entity is its primary source of moral or physical strength, power or resistance.

Joe Strange, Centers of Gravity & Critical Vulnerabilities, 1996.

Government

Military

People

Economy

Alliances

Etc.

Which are the critical capabilities?

Are the critical requirements of these capabilities satisfied?

If not, eliminate the candidate.

If yes, do these capabilities have any vulnerability?

• Approach to center of gravity analysis based on the concepts ofcritical capabilities, critical requirements and critical vulnerabilities, which have been recently adopted into the joint military doctrine.

• Application to current war scenarios (e.g. War on terror 2003, Iraq 2003) with state and non-state actors (e.g. Al Qaeda).

Identify potential primary sources of moral or physical

strength, power and resistance from:

Test each identified COG candidate to determine whether it has all the necessary critical

capabilities:

Identify COG candidates Test COG candidates

First computational approach to COG analysis




Overview



Acknowledgements



S1

S11a

S1n

S11b1 S11bm

T11bmT11b1

T1nT11a

…

…

T1

Q1

S11bT11b

A1n S11A11

……

A11b1 A11bm

S11bQ11b

Let T1 be the problem solving task to be performed.

Finding a solution is an iterative process where, at each step, we consider some relevant information that leads us to reduce the current task to a simpler task or to several simpler tasks.

The question Q associated with the current task identifies the type of information to be considered.

The answer A identifies that piece of information and leads us to the reduction of the current task.

A complex problem solving task is performed by:

• successively reducing it to simpler tasks;

• finding the solutionsof the simplest tasks;

• successively composing these solutions until the solution to the initial task is obtained.

Problem Solving: Task Reduction

Identify and test a strategic COG candidate for Allied_Forces_1943 which is a multi_member_force

What type of strategic COG candidate should I consider for this multi_member_force?

Identify and test a strategic COG candidate corresponding to a member of the Allied_Forces_1943

I consider a candidate corresponding to a member of the multi_member_force

Therefore we need to

US_1943

Identify and test a strategic COG candidate for US_1943

Which is a member of Allied_Forces_1943?


Identify and test a strategic COG candidate for Sicily_1943

We need to

Which is an opposing_force in the Sicily_1943 scenario?

Allied_Forces_1943

Identify and test a strategic COG candidate for Allied_Forces_1943


Is Allied_Forces_1943 a single_member_force or a multi_member_force?

Allied_Forces_1943 is a multi_member_force


COG Analysis:World War II at the time of Sicily 1943




Overview



Acknowledgements



Knowledge Base: Object Ontology + Rules

A hierarchical representation of the objects and types of

objects.

A hierarchical representation of the types of

features.

has_as_controlling_leaderDomain: agentRange: person

has_as_religious_leaderDomain: governing_bodyRange: person

has_as_military_leaderDomain: governing_bodyRange: person

has_as_god_kingDomain: governing_bodyRange: person

has_as_monarchDomain: governing_bodyRange: person

has_as_political_leaderDomain: governing_bodyRange: person

has_as_head_of_stateDomain: governing_bodyRange: person

has_as_head_of_governmentDomain: governing_bodyRange: person

has_as_commander_in_chiefDomain: forceRange: person

Object Ontology<object>

Scenario Strategic_COG_relevant_factor

Established_governing_body

Resource_or_ infrastructure_element

Psychosocial_factor

Geographic_factor

Historic_factor

Military_factor

Political_factor Other_relevant_factor

Civilization_factor

War_scenario

Ad_hoc_governing_body

Other_type_of_governing_body

Demographic_factor

Agent Force_goal

International_factor

Other_political_factor

Controlling_element

Group_governing_body

Governing_body

State_government

Economic_factor

Operation_other_than_war

Question Which is a member of ?O1 ?Answer ?O2

INFORMAL STRUCTURE

IFIdentify and test a strategic COG candidate corresponding to a member of the ?O1

THENIdentify and test a strategic COG candidate for ?O2

US_1943


We need to Identify and test a strategic COG candidate corresponding to a member of the Allied_Forces_1943


EXAMPLE OF REASONING STEP

FORMAL STRUCTURE

IFIdentify and test a strategic COG candidate corresponding to a member of a force

The force is ?O1

THENIdentify and test a strategic COG candidate for a force

The force is ?O2

Plausible Upper Bound Condition ?O1 is multi_member_force

has_as_member ?O2 ?O2 is force

Plausible Lower Bound Condition ?O1 is equal_partners_multi_state_alliance

has_as_member ?O2 ?O2 is single_state_force


Knowledge Base: Object Ontology + Rules

LEARNED RULE

Universe of Instances

Concept

Plausible Upper Bound

Plausible Lower Bound

Learnable knowledge representation

Use of the object ontology as an incomplete and evolving generalization hierarchy.

Plausible version space (PVS)Use of plausible version spaces to represent and use partially learned knowledge:• Rules with PVS

conditions• Tasks with PVS

conditions• Object features with

PVS concept• Task features with

PVS concept

Feature Domain: PVS concept Range: PVS concept




Overview



Acknowledgements



Integrated modeling, learning and problem solving

Input Task

Generated Reduction

Mixed-Initiative Problem Solving

Ontology + Rules

Reject ReductionAccept Reduction

Rule Generalization

Rule Specialization

Modeling

Rule Learning

Specified Reduction

We need to

US_1943





Provides an example

1

Rule_15Learns

2

Rule_15

?

Applies

Germany_1943

Identify and test a strategic COG candidate for Germany_1943

Which is a member of European_Axis_1943?


3

Accepts the example

4 Rule_15Refines

5We need to

Identify and test a strategic COG candidate corresponding to a member of the European_Axis_1943

…

Disciple uses the learned rules in problem solving, and refines them based on expert’s feedback.

Modeling Learning

Problem Solving Refining

Rule learning method

Example of atask reduction

step

Plausible version space rule

analogy

PLB

PUB

Knowledge Base

Incompleteexplanation

Analogy and HintGuided Explanation

Analogy-basedGeneralization

Find an explanation of why the example is correct

US_1943has_as_member

Allied_Forces_1943

The explanation is the best possible approximation of the question and the answer, in the object ontology.

US_1943



We need to



Identify and test a strategic COG candidate corresponding to a member of a force

The force is Allied_Forces_1943

Identify and test a strategic COG candidate for a force

The force is US_1943

We need to


Generate the PVS rule

Condition?O1 is Allied_Forces_1943

has_as_member ?O2?O2 is US_1943

Most general generalization

IFIdentify and test a strategic COG candidate corresponding to a member of a force

The force is ?O1

THENIdentify and test a strategic COG candidate for a force

The force is ?O2

Plausible Upper Bound Condition ?O1 is multi_member_force

has_as_member ?O2 ?O2 is force

Plausible Lower Bound Condition ?O1 is equal_partners_multi_state_alliance

has_as_member ?O2 ?O2 is single_state_force

explanation?O1 has_as_member ?O2

Most specific generalization

US_1943has_as_member

Allied_Forces_1943

Rewriteas

has_as_member domain: multi_member_force range: force

Rule refinement method

Failureexplanation

PVSRule

Example of task reductionsgenerated by the agent

Incorrectexample

Correctexample

Learning fromExplanations

Learning by AnalogyAnd Experimentation

Learning from Examples

Knowledge BaseIF<task>

THEN<subtask 1>…<subtask m>

PVS Condition<condition 1>

PVS Except When Condition<condition 2>…PVS Except When Condition<condition n>




Overview



Acknowledgements


Initial modeling

Ontology specification

Ontology import

Ontology development

Initialobject

ontology

Force

<object>

Scenario

Strategic_goal

Force_goal

Strategic_COG_relevant_factor

Demographic_factor


Psychosocial_factorEconomic_factor

Geographical_factor

Historical_factor

Military_factor

Political_factor

Other_relevant_factor

Operational_goal

Civilization_factor

resource_ or_ infrastructure_element

Problemspecification

Modeling the problem solving process of the subject matter

expert and development of the object ontology of the agent.

Scenarios(elicited)

Generalrules

(learned)has_as_pr im ary_

force_elem ent

ha s asoppo sing

force

Sic ily_1943

“W W II Allied invasiono f Sicily in 1943”

brief_descr iption

Anglo_allies_1943

US_19 43

Brita in_194 3

com ponent_state

compone nt_state

Allied_forces_operation_Husky

type_of_opera tions

has_as_subg roup

US_7 th _Ar my_(Fo rce_ 343)

“combined and joint operations”

has_a s_subgroup

Br_ 8th_A rm y_(Force_545)

has_as_subgroupWe stern_Naval_TFhas_as_subgro up

Eastern_N aval_TFhas_as_subgroup

US_9 th_Air_Fo rcehas_a s_subgroup

Northwest_A frica_A ir_Fo rce

instance_of

instance_of

instance_ofinstance_of

instance_ of

insta nce_of

instance_of

instance_of

instance_of

insta nce_of

insta nce_of

insta nce_of

ha s_as_ industrial_factor

IF the task isIdentify the strategic COG candidates with respect to theindustrial civilization of a state which is a member of a force

The state is ?O2The force is ?O1

THENConclude that an economic factor is a strategic COGcandidate for a state which is a member of a force

The state is ?O2The force is ?O1The economic factor is ?O3

Plausible Upper Bound Condition?O1 IS Force?O2 IS Force

has_as_industrial_factor ?O3?O3 IS Industrial_factor

is_a_major_generator_of ?O4?O4 IS Strategically_essential_goods_o_materiel

Plausible Lower Bound Condition?O1 IS Anglo_allies_1943?O2 IS US_1943

has_as_industrial_factor ?O3?O3 IS Industrial_capacity_of_US_1943

is_a_major_generator_of ?O4?O4 IS War_materiel_and_transports_of_US_1943

explanation?O2 has_as_industrial_factor ?O3?O3 is_a_major_generator_of ?O4?O4 IS strategically_essential_goods_or_materiel














































































































































































































































































































































































































































Force

<object>

Scenario

Strategic_goal

Force_goal

Strategic_COG_relevant_factor

Demographic_factor


Psychosocial_factorEconomic_factor

Geographical_factor

Historical_factor

Military_factor

Political_factor

Other_relevant_factor

Operational_goal

Civilization_factor

resource_ or_ infrastructure_element

Scenario specification

Modeling

Task and rule learning

Task, rule, ontology refinement

Problem solving

Objectontology(refined)

Teaching of the agent by the subject

matter expert.

Agent Development Methodology

012345678

Stro

ngly

Dis

agre

e

Dis

agre

e

Neu

tral

Agr

ee

Stro

ngly

Agr

ee

012345678

Stro

ngly

Dis

agre

e

Dis

agre

e

Neu

tral

Agr

ee

Stro

ngly

Agr

ee

DiscipleAgent KB

Problem

solving

Disciple was taught based on the expertise of Prof. Comello in center of gravity analysis.

Disciple helps the students to perform a center of gravity analysis of an assigned war scenario.

Teaching

Learning

012345678

Stro

ngly

Dis

agre

e

Dis

agre

e

Neu

tral

Agr

ee

Stro

ngly

Agr

ee

The use of Disciple is an assignment that is well suited to the course's learning objectives

Disciple should be used in future versions of this course

Use of Disciple at the US Army War College

319jw Case Studies in Center of Gravity Analysis

Disciple helped me to learn to perform a strategic COG

analysis of a scenario

Global evaluations of Disciple by officers from the Spring 03 course

0123456789

Stro

ngly

Dis

agre

e

Dis

agre

e

Neu

tral

Agr

ee

Stro

ngly

Agr

ee

0123456789

Stro

ngly

Dis

agre

e

Dis

agre

e

Neu

tral

Agr

ee

Stro

ngly

Agr

ee

0123456789

Stro

ngly

Dis

agre

e

Dis

agre

e

Neu

tral

Agr

ee

Stro

ngly

Agr

ee

Use of Disciple at the US Army War College

589jw Military Applications of Artificial Intelligence course

Students teach Disciple their COG analysis expertise,

using sample scenarios (Iraq 2003, War on terror 2003, Arab-Israeli 1973)

Students test the trained

Disciple agent based on a

new scenario (North Korea

2003)

I think that a subject matter expert can use Disciple to build an agent, with limited assistance from a knowledge engineer

Spring 2001COG identification

Spring 2002COG identification

and testing

Spring 2003COG testing based on

critical capabilities

Global evaluations of Disciple by officers during three experiments

Extended KB

stay informedbe irreplaceable

communicate be influential

Integrated KB

Initial KB

have supportbe protected

be driving force

432 concepts and features, 29 tasks, 18 rulesFor COG identification for leaders

37 acquired concepts andfeatures for COG testing

COG identification and testing (leaders)

Domain analysis and ontology development (KE+SME)

Parallel KB development (SME assisted by KE)

KB merging (KE)

Knowledge Engineer (KE)

All subject matter experts (SME)

DISCIPLE-COG DISCIPLE-COG DISCIPLE-COG DISCIPLE-COG DISCIPLE-COG

Training scenarios:Iraq 2003

Arab-Israeli 1973War on Terror 2003

Team 1 Team 2 Team 3 Team 4 Team 5

5 features10 tasks10 rules

Learned features, tasks, rules

14 tasks14 rules


35 tasks33 rules


Unified 2 features Deleted 4 rules Refined 12 rulesFinal KB:+9 features 478 concepts and features+105 tasks 134 tasks+95 rules 113 rules

DISCIPLE-COG

Testing scenario:North Korea 2003

Correctness = 98.15%

5h 28min average training time / team3.53 average rule learning rate / team

Parallel development and merging of KBs

Disciple-WA (1997-1998): Estimates the best plan of working around damage to a transportation infrastructure, such as a

damaged bridge or road.

River bedSite 106

Left bankSite 105Right bank

Site 107

Site 103:cross-sectionDamage 200: destroyed bridge

Bridge/Rivergap = 25 meters

Near approach(Right approach)Site 108

Far approach(Left approach)Site 104

River bedSite 106


Site 107





River bedSite 106


Site 107





Demonstrated that a knowledge engineer can use Disciple to rapidly build and update a knowledge base capturing knowledge from military engineering manuals and a set of sample solutions provided by a subject matter expert.

Disciple-COA (1998-1999): Identifies strengths and weaknesses in a Course of Action, based on the principles of war and

the tenets of Army operations.

Demonstrated the generality of its learning methods that used an object ontology created by another group (TFS/Cycorp).

Demonstrated that a knowledge engineer and a subject matter expert can jointly teach Disciple.

Other Disciple agents

Mission: BLUE-BRIGADE2 attacks to penetrate RED-MECH-REGIMENT2 at 130600 Aug in order to enable the completion of seize OBJ-SLAM by BLUE-ARMOR-BRIGADE1.

Close: BLUE-TASK-FORCE1, a balanced task force (MAIN EFFORT) attacks to penetrate RED-MECH-COMPANY4, then clears RED-TANK-COMPANY2 in order to enable the completion of seize OBJ-SLAM by BLUE-ARMOR-BRIGADE1. BLUE-TASK-FORCE2, a balanced task force (SUPPORTING EFFORT 1) attacks to fix RED-MECH-COMPANY1 and RED-

MECH-COMPANY2 and RED-MECH-COMPANY3 in order to prevent RED-MECH-COMPANY1 and RED-MECH-COMPANY2 and RED-MECH-COMPANY3 from interfering with conducts of the MAIN-EFFORT1, then clears RED-MECH-COMPANY1 and RED-MECH-COMPANY2 and RED-MECH-COMPANY3 and RED-TANK-COMPANY1. …

Reserve: The reserve, BLUE-MECH-COMPANY8, a mechanized infantry company, follows Main Effort, and is prepared to reinforce ) MAIN-EFFORT1.

Security: SUPPORTING-EFFORT1 destroys RED-CSOP1 prior to begin moving across PL-AMBER by MAIN-EFFORT1 in order to prevent RED-MECH-REGIMENT2 from observing MAIN-EFFORT1. …

Deep: Deep operations will destroy RED-TANK-COMPANY1 and RED-TANK-COMPANY2 and RED-TANK-COMPANY3.

Rear: BLUE-MECH-PLT1, a mechanized infantry platoon secures the brigade support area.

Fires: Fires will suppress RED-MECH-COMPANY1 and RED-MECH-COMPANY2 and RED-MECH-COMPANY3 and RED-MECH-COMPANY4 and RED-MECH-COMPANY5 and RED-MECH-COMPANY6.

End State: At the conclusion of this operation, BLUE-BRIGADE2 will enable accomplishing conducts forward passage of lines through BLUE-BRIGADE2 by BLUE-ARMOR-BRIGADE1. MAIN-EFFORT1 will complete to clear RED-MECH-COMPANY4 and RED-TANK-COMPANY2. SUPPORTING-EFFORT1 will complete to clear RED-MECH-COMPANY1 and RED-MECH-COMPANY2 and RED-MECH-COMPANY3 and RED-TANK-COMPANY1. SUPPORGING-EFFORT2 will complete to clear RED-MECH-COMPANY5 and RED-MECH-COMPANY6 and RED-TANK-COMPANY3.




Overview



Acknowledgements



LearningAgent

Modeling

Non-disruptiveLearning

Ontology Elicitation

Rule & Ontology Learning

Rule & OntologyRefining

User ModelLearning

ExceptionHandling

KBMaintenance

Implicit reasoning ofhuman expert Explicit reasoning in

natural language Ontology extensions

Le

arn

ed

rule

s,

on

tolo

gy

Re

fine

d ru

les

, o

nto

log

y

User modelCases, rules

Ru

les

w/o

e

xc

ep

tio

ns

1. Multistrategy teaching and learning 2. Mixed

-initiative p

rob

lem so

lving

and

learnin

g

3. Autonomous (and interactive) multistrategy learning

• Analogy based methods• Explanation based methods• Natural Language based methods• Abstraction based methods

• Plausible version spaces• Learning from instruction• Learning from examples• Learning from explanations• Learning by analogy

• Mixed-initiative learning• Routine, innovative, inventive, and creative reasoning

• Automatic inductive learning• Case-based learning• Abductive learning• Ontology discovery • KB optimization• KB maintenance

Life-Long Continuous Agent Learning4.

KB

mai

nte

nan

ce a

nd

op

tim

izat

ion




Overview



Acknowledgements



This research was sponsored by the Defense Advanced Research Projects Agency, Air Force Research Laboratory, Air Force Material Command, USAF under agreement number F30602-00-2-0546, by the Air Force Office of Scientific Research under grant number F49620-00-1-0072 and by the US Army War College.

Acknowledgements

Documents

Learning Agents Center George Mason University Symposium on Reasoning and Learning in Cognitive Systems Stanford, CA, 20-21 May 2004 Gheorghe Tecuci with