Learning Agents Center George Mason University Computer Science Department Partners Day Symposium May 4, 2004 Gheorghe Tecuci, Mihai Boicu, Dorin Marcu,

Learning Agents CenterGeorge Mason University

Computer Science Department Partners Day SymposiumMay 4, 2004

Gheorghe Tecuci, Mihai Boicu, Dorin Marcu,Bogdan Stanescu, Cristina Boicu, Marcel Barbulescu

Knowledge Representation, Reasoning, and Learning

Overview

Experiments of Agent Development and Use

Long Term Research Vision

Acknowledgements

Research Problem, Approach, and Application

The knowledge engineer attempts to understand how the subject matter expert reasons and solves problems and then encodes the acquired expertise into the agent's knowledge base.

This modeling and representation of expert’s knowledge is long, painful and inefficient (known as the “knowledge acquisition bottleneck”).

How are agents built and why it is hard

KnowledgeEngineer

DomainExpert

Knowledge Base

Inference Engine

Intelligent Agent

ProgrammingDialog

Results

The expert teachesthe agent how to perform

various tasks in a way thatresembles how the expert

would teach a person.

1. Mixed-initiative problem solving

2. Teaching and learning

3. Multistrategy learning In

terf

ace

ProblemSolving

Learning

Ontology+ Rules

Research Problem and Approach

Research Problem: Elaborate a theory, methodology and family of systems for the development of knowledge-base agents by subject matter experts, with limited assistance from knowledge engineers.

Approach: Develop a learning agent that can be taught directly by a subject matter expert while solving problems in cooperation.

The agent learnsfrom the expert,

building, verifyingand improving itsknowledge base

Knowledge bases and agent development by subject matter experts, using learning agent technology. Experiments in the USAWC courses.

Formalization ofthe Center of Gravity(COG) analysis process

319jw Case Studies inCenter of Gravity Analysis

Use of Disciple in a sequence of two joint warfighting courses

589jw Military Applications of Artificial Intelligence

Students developedscenarios

Students developed

agents

Extended KB

stay informedbe irreplaceable

communicate be influential

Integrated KB

Initial KB

have supportbe protected

be driving force

432 concepts and features, 29 tasks, 18 rulesFor COG identification for leaders

37 acquired concepts andfeatures for COG testing

COG identification and testing (leaders)

Domain analysis and ontology development (KE+SME)

Parallel KB development (SME assisted by KE)

KB merging (KE)

Knowledge Engineer (KE)

All subject matter experts (SME)

DISCIPLE-COG DISCIPLE-COG DISCIPLE-COG DISCIPLE-COG DISCIPLE-COG

Training scenarios:Iraq 2003

Arab-Israeli 1973War on Terror 2003

Team 1 Team 2 Team 3 Team 4 Team 5

5 features10 tasks10 rules

Learned features, tasks, rules

14 tasks14 rules


35 tasks33 rules


Unified two featuresDeleted 4 incomplete rulesRefined 11 rules+9 features 478 concepts and features+105 tasks 134 tasks+95 rules 113 rules

DISCIPLE-COG

Testing scenario:North Korea 2003Correctness = 98.15%

Completeness = 89.33%

2.5 examples/rule5.47 hours average training time

Identify the strategic COG candidates for the Sicily_1943 scenario

Anglo_allies_1943

Identify the strategic COG candidates for Anglo_allies_1943

Which is an opposing force in the Sicily_1943 scenario?

Is Anglo_allies_1943 a single member force or a multi-member force?

Anglo_allies_1943 is a multi-member force

Identify the strategic COG candidates for the Anglo_allies_1943 which is a multi-member force

What type of strategic COG candidates should I consider for a multi-member force?

Identify the strategic COG candidates corresponding to the multi-member nature of the Anglo_allies_1943

I consider the candidates corresponding to the multi-member nature of the force

What type of strategic COG candidates should I consider for the multi-member nature of the force?

I consider the relationships between the members of the force

I consider the type of operations being conducted by the members of the force

Identify the strategic COG candidates with respect to the type of operations being conducted by the members of the Anglo_allies_1943

Which is the primary force element that will conduct the campaign for Anglo_allies_1943?

Allied_forces_operations_Husky

Is Allied_forces_operations_Husky made up of a true single group or are there subgroups?

Allied_forces_operations_Husky is made up of several subgroups

Identify the strategic COG candidates with respect to the type of operations being conducted by Allied_forces_operations_Husky

Synergistic collaboration and transition to the USAWCGeorge Mason University - US Army War College

ArtificialIntelligenceResearch

Mili

tary

Stra

tegy

Rese

arch

Military

Education

& PracticeDisciple

Government

Military

People

Economy

Alliances

Etc.

Which are the critical capabilities?

Are the critical requirements of these capabilities satisfied?

If not, eliminate the candidate.

If yes, do these capabilities have any vulnerability?

Application to current war scenarios (e.g. War on terror, Iraq) with state and non-state actors (e.g. Al Qaeda).

Identify potential primary sources of moral or physical

strength, power and resistance from:

Test each identified COG candidate to determine whether it has all the necessary critical

capabilities:

Identify COG candidates Test COG candidates

Sample Domain: Center of Gravity Analysis

Centers of Gravity: Primary sources of moral or physical strength, power or resistance of the opposing forces in a conflict.

Problem Solving Approach: Task Reduction

S1

S11a

S1n

S11b1 S11bm

T11bmT11b1

T1nT11a

…

…

T1

Q1

S11bT11b

A1nS11A11

……

A11b1 A11bm

S11bQ11b

President Roosevelt is a strategic COG candidate that can be eliminated

Test whether President Roosevelt has the critical capability to be protected

Test whether President Roosevelt has the critical capability to stay informed

Test whether President Roosevelt has the critical capability to communicate

President Roosevelt has the critical capability to communicate through executive orders, through military orders, and through the Mass Media of US 1943. These communication

means have no significant vulnerabilities

Test whether President Roosevelt has the critical capability to have support

Test whether President Roosevelt has the critical

capability to be a driving force

Test whether President Roosevelt has the critical capability to be influential

Test whether President Roosevelt is a viable strategic COG candidate

Does President Roosevelt have all the necessary critical capabilities?

No.The necessary critical capabilities are: be protected, stay informed, communicate,

be influential, be a driving force, have support and be irreplaceable

Which are the critical capabilities that President Roosevelt should have to be a COG candidate?

Test whether President Roosevelt has the critical

capability to be irreplaceable

President Roosevelt has the critical capability to be protected. President Roosevelt is protected by US Service 1943 which has no significant vulnerability

President Roosevelt has the critical capability to stay informed. President Roosevelt receives essential intelligence from intelligence agencies which have no significant

vulnerability

President Roosevelt has the critical capability to be influential because he is the head of the government of US 1943, the commander in chief of the military of US 1943, and is a trusted

leader who can use the Mass Media of US 1943. These influence means have no significant vulnerabilities.

President Roosevelt has the critical capability to have support because he is the head of a democratic government with a history of good decisions, a trusted commander in chief of the military, and the people are willing to make sacrifices for unconditional surrender of

European Axis. The means to secure continuous support have no significant vulnerability.

President Roosevelt has the critical capability to be a driving force. The main reason for President Roosevelt to pursue the goal of unconditional surrender of European Axis is “preventing separate peace by the members of the Allied Forces”. Also, “the western

democratic values” provides President Roosevelt with determination to persevere in this goal. There is no significant vulnerability in the reason and determination.

President Roosevelt does not have the critical capability to be irreplaceable. US 1943 would maintain the goal of unconditional surrender of European Axis irrespective of its leader because “the goal was established and the country was committed to it”. There is no

significant vulnerability resulted from the replacement of President Roosevelt

A complex problem solving task is performed by:

• successively reducing it to simpler tasks;

• finding the solutions of the simplest tasks;

• successively composing these solutions until the solution to the initial task is obtained.

Object Ontology Reduction Rules

Composition Rules

Knowledge Base

Question Which is a member of ?O1 ?Answer ?O2

IFIdentify and test a strategic COG candidate corresponding to a member of the ?O1

THENIdentify and test a strategic COG candidate for ?O2

US_1943

Which is a member of Allied_Forces_1943?

We need to

Identify and test a strategic COG candidatecorresponding to a member of the Allied_Forces_1943

Therefore we need to EXAMPLE OF REASONING

STEP

IFIdentify and test a strategic COG candidate corresponding to a member of a force

The force is ?O1

THENIdentify and test a strategic COG candidate for a force

The force is ?O2

Plausible Upper Bound Condition ?O1 is multi_member_force

has_as_member ?O2 ?O2 is force

Plausible Lower Bound Condition ?O1 is equal_partners_multi_state_alliance

has_as_member ?O2 ?O2 is single_state_force

Identify and test a strategic COG candidate for US_1943

Problem Solving and Learning

LEARNED RULE

FORMAL STRUCTURE

has as member

force

multi group force single group force single state force

US 1943

multi state force

single member forcemulti member force

Allied Forces 1943

multi state alliance multi state coalition

equal partners multi state alliance

dominant partner multi state alliance

. . .

. . .

. . .

INFORMAL STRUCTURE

ONTOLOGY FRAGMENT

012345678

Stro

ngly

Dis

agre

e

Dis

agre

e

Neu

tral

Agr

ee

Stro

ngly

Agr

ee

012345678

Stro

ngly

Dis

agre

e

Dis

agre

e

Neu

tral

Agr

ee

Stro

ngly

Agr

ee

DiscipleAgent KB

Problem

solving

Disciple was taught based on the expertise of Prof. Comello in center of gravity analysis.

Disciple helps the students to perform a center of gravity analysis of an assigned war scenario.

Teaching

Learning

012345678

Stro

ngly

Dis

agre

e

Dis

agre

e

Neu

tral

Agr

ee

Stro

ngly

Agr

ee

The use of Disciple is an assignment that is well suited to the course's learning objectives

Disciple should be used in future versions of this course

Use of Disciple at the US Army War College

319jw Case Studies in Center of Gravity Analysis

Disciple helped me to learn to perform a strategic COG

analysis of a scenario

Global evaluations of Disciple by officers from the Spring 03 course

0123456789

Stro

ngly

Dis

agre

e

Dis

agre

e

Neu

tral

Agr

ee

Stro

ngly

Agr

ee

0123456789

Stro

ngly

Dis

agre

e

Dis

agre

e

Neu

tral

Agr

ee

Stro

ngly

Agr

ee

0123456789

Stro

ngly

Dis

agre

e

Dis

agre

e

Neu

tral

Agr

ee

Stro

ngly

Agr

ee

Use of Disciple at the US Army War College

589jw Military Applications of Artificial Intelligence course

Students teach Disciple their COG analysis expertise,

using sample scenarios (Iraq 2003, War on terror 2003, Arab-Israeli 1973)

Students test the trained

Disciple agent based on a

new scenario (North Korea

2003)

I think that a subject matter expert can use Disciple to build an agent, with limited assistance from a knowledge engineer

Spring 2001COG identification

Spring 2002COG identification

and testing

Spring 2003COG testing based on

critical capabilities

Global evaluations of Disciple by officers during three experiments

Extended KB

stay informedbe irreplaceable

communicate be influential

Integrated KB

Initial KB

have supportbe protected

be driving force

432 concepts and features, 29 tasks, 18 rulesFor COG identification for leaders

37 acquired concepts andfeatures for COG testing

COG identification and testing (leaders)

Domain analysis and ontology development (KE+SME)

Parallel KB development (SME assisted by KE)

KB merging (KE)

Knowledge Engineer (KE)

All subject matter experts (SME)

DISCIPLE-COG DISCIPLE-COG DISCIPLE-COG DISCIPLE-COG DISCIPLE-COG

Training scenarios:Iraq 2003

Arab-Israeli 1973War on Terror 2003

Team 1 Team 2 Team 3 Team 4 Team 5


Learned features, tasks, rules

14 tasks14 rules


35 tasks33 rules


Unified 2 features Deleted 4 rules Refined 12 rulesFinal KB:+9 features 478 concepts and features+105 tasks 134 tasks+95 rules 113 rules

DISCIPLE-COG

Testing scenario:North Korea 2003

Correctness = 98.15%

5h 28min average training time / team3.53 average rule learning rate / team

Parallel development and merging of knowledge bases

Disciple-WA (1997-1998): Estimates the best plan of working around damage to a transportation infrastructure, such as a

damaged bridge or road.

River bedSite 106

Left bankSite 105Right bank

Site 107

Site 103:cross-sectionDamage 200: destroyed bridge

Bridge/Rivergap = 25 meters

Near approach(Right approach)Site 108

Far approach(Left approach)Site 104

River bedSite 106


Site 107





River bedSite 106


Site 107





Demonstrated that a knowledge engineer can use Disciple to rapidly build and update a knowledge base capturing knowledge from military engineering manuals and a set of sample solutions provided by a subject matter expert.

Disciple-COA (1998-1999): Identifies strengths and weaknesses in a Course of Action, based on the principles of war and

the tenets of army operations.

Demonstrated the generality of its learning methods that used an object ontology created by another group (TFS/Cycorp).

Demonstrated that a knowledge engineer and a subject matter expert can jointly teach Disciple.

Other Disciple agents

Mission: BLUE-BRIGADE2 attacks to penetrate RED-MECH-REGIMENT2 at 130600 Aug in order to enable the completion of seize OBJ-SLAM by BLUE-ARMOR-BRIGADE1.

Close: BLUE-TASK-FORCE1, a balanced task force (MAIN EFFORT) attacks to penetrate RED-MECH-COMPANY4, then clears RED-TANK-COMPANY2 in order to enable the completion of seize OBJ-SLAM by BLUE-ARMOR-BRIGADE1. BLUE-TASK-FORCE2, a balanced task force (SUPPORTING EFFORT 1) attacks to fix RED-MECH-COMPANY1 and RED-

MECH-COMPANY2 and RED-MECH-COMPANY3 in order to prevent RED-MECH-COMPANY1 and RED-MECH-COMPANY2 and RED-MECH-COMPANY3 from interfering with conducts of the MAIN-EFFORT1, then clears RED-MECH-COMPANY1 and RED-MECH-COMPANY2 and RED-MECH-COMPANY3 and RED-TANK-COMPANY1. …

Reserve: The reserve, BLUE-MECH-COMPANY8, a mechanized infantry company, follows Main Effort, and is prepared to reinforce ) MAIN-EFFORT1.

Security: SUPPORTING-EFFORT1 destroys RED-CSOP1 prior to begin moving across PL-AMBER by MAIN-EFFORT1 in order to prevent RED-MECH-REGIMENT2 from observing MAIN-EFFORT1. …

Deep: Deep operations will destroy RED-TANK-COMPANY1 and RED-TANK-COMPANY2 and RED-TANK-COMPANY3.

Rear: BLUE-MECH-PLT1, a mechanized infantry platoon secures the brigade support area.

Fires: Fires will suppress RED-MECH-COMPANY1 and RED-MECH-COMPANY2 and RED-MECH-COMPANY3 and RED-MECH-COMPANY4 and RED-MECH-COMPANY5 and RED-MECH-COMPANY6.

End State: At the conclusion of this operation, BLUE-BRIGADE2 will enable accomplishing conducts forward passage of lines through BLUE-BRIGADE2 by BLUE-ARMOR-BRIGADE1. MAIN-EFFORT1 will complete to clear RED-MECH-COMPANY4 and RED-TANK-COMPANY2. SUPPORTING-EFFORT1 will complete to clear RED-MECH-COMPANY1 and RED-MECH-COMPANY2 and RED-MECH-COMPANY3 and RED-TANK-COMPANY1. SUPPORGING-EFFORT2 will complete to clear RED-MECH-COMPANY5 and RED-MECH-COMPANY6 and RED-TANK-COMPANY3.

Disciple’s vision on the future of software development

MainframeComputers

Software systems developed and used by computer experts

PersonalComputers

Software systems developedby computer experts

and used by persons thatare not computer experts

LearningAgents

Software systems developed and used by persons that are

not computer experts

DISCIPLE

Inte

rfac

e

ProblemSolving

Learning

Ontology+ Rules

Vision on the use of Disciple in Education

teachesDiscipleAgent KB

The expert/teacher teaches Disciple through examples and explanations, in a way that is similar to how the expert would teach a student.



…

Disciple tutors the student in a way that is similar to how the expert/teacher has taught it.


This research was sponsored by the Defense Advanced Research Projects Agency, Air Force Research Laboratory, Air Force Material Command, USAF under agreement number F30602-00-2-0546, by the Air Force Office of Scientific Research under grant number F49620-00-1-0072 and by the US Army War College.

Acknowledgements

Documents

Learning Agents Center George Mason University Computer Science Department Partners Day Symposium May 4, 2004 Gheorghe Tecuci, Mihai Boicu, Dorin Marcu,