21 September 2005 Gheorghe Tecuci Learning Agents Center and Computer Science Department School of Information Technology and Engineering George Mason

21 September 2005

Gheorghe TecuciLearning Agents Center

and Computer Science DepartmentSchool of Information Technology and Engineering

George Mason University

Personal Cognitive Assistant for Intelligence Analysis

Virtual Experts for Multi-domain Collaborative Planning

Learning Agents Center: Research Vision

Research Issues for Learning Agents

Agent for Course of Action Critiquing

Overview

Agents for Centers of Gravity and Critical Vulnerabilities

Final Remarks

• Conducts fundamental and experimental research on the development of knowledge-based learning and problem solving agents.

• Supports teaching in the areas of intelligent agents, machine learning, knowledge acquisition, artificial intelligence and its applications.

• Develops the Disciple theory, methodology and agent shells for building agents that can be taught how to solve problems by subject matter experts.

http://lac.gmu.edu

Mission

Basic Research

Tools

Applications

Transitions

Turing, A.M. (1950). Computing machinery and intelligence. Mind, 59, 433-460.

Building an intelligent machine by programming is too difficult.

“Instead of trying to produce a programme to simulate the adult mind, why not rather try to produce one which simulates the child's? If this were then subjected to an appropriate course of education one would obtain the adult brain.”

Teaching as Alternative to Programming

How are Expert Systems Built and Why it is Hard

Edward Feigenbaum, 1993: Rarely does a technology arise that offers such a wide range of important benefits.

KnowledgeEngineer

Subject MatterExpert

Knowledge Base

Inference Engine

Expert System

ProgrammingDialog

Results

Research Problem and Approach

The expert teachesthe agent to perform various tasks in a way that resembles

how the expert would teach a person.

The agent learnsfrom the expert,

building, verifyingand improving itsknowledge base

Inte

rfac

eProblemSolving

Learning

Ontology+ Rules

Disciple

Disciple’s Vision on the Future of Software Development

MainframeComputers

PersonalComputers

LearningAgents

The expert teachesthe agent to perform various tasks in a way that resembles

how the expert would teach a person.

The agent learnsfrom the expert,

building, verifyingand improving itsknowledge base

Inte

rfac

e

ProblemSolving

Learning

Ontology+ Rules

Disciple

Vision on the Use of Disciple in Education

teachesDiscipleAgent KB



…


2005, Learning Agents Center






Overview


Final Remarks


The Overall Architecture of a Disciple Agent

Problem SolvingEngines

Disciple

SubjectMatterExpert

Dialog Knowledge Acquisitionand Learning Engines

Ontology

Rules

Knowledge Base

RULE x,y,z OBJECT, (ON x y) & (ON y z) (ON x z)

ONCUP1 BOOK1 ON TABLE1

CUP BOOK TABLE

INSTANCE-OF

OBJECT

SUBCLASS-OF

ONCUP1 BOOK1 ON TABLE1

CUP BOOK TABLE

INSTANCE-OF

OBJECT

SUBCLASS-OF

ONTOLOGY

Knowledge Base = Ontology + Rules

ONTOLOGY FRAGMENT

Main condition?O1 is PhD_advisor

has_as_employer ?O4 has_as_position ?O5

?O2 is PhD_student?O3 is research_area?O4 is university?O5 is tenured_position

Except when condition?O1 is person

is_likely_to_move_to ?O6?O6 is employer

IF: Determine whether ?O1 can be a PhD advisor for ?O2 in ?O3.

THEN: Determine whether ?O1 would be a good PhD advisor for ?O2 in ?O3.

REASONING RULE

Determine whether John Smith can be a PhD advisor for Tom Even in Artificial Intelligence.

PROBLEM SOLVING

TASK

Ph.D. student

facultymemberstaff

member

professor

studentuniversityemployee

person

subconcept-of

subconcept-of

subconcept-of subconcept-of

subconcept-of

M.S. student

B.S. studentinstructor

graduatestudent

undergraduatestudent

fullprofessor

associateprofessor

assistantprofessor

subconcept-of

subconcept-of

PhD_advisor

Model the reasoning

of SME

Createobject

ontology

Definereasoning

rules

Verify and update rules

KE

SME

Traditionally

KE

Agent

SME Agent

SME

Specifyinstances

and features

Learnontologicalelements

Import anddevelop initial

ontology

Agent

Learnreasoning

rules

SME Agent

Define andexplain

examples

SME

AgentSME Agent

Critiqueexamples

Refinerules

Explaincritiques

SME Agent

Develop reasoning

trees

SMEKE

Instruct SME to explain reasoning

With Disciple

Main Idea of the Disciple Approach

Determine whether John Smith can be a PhD advisor for Tom Even in Artificial Intelligence.


Agent Architecture for Generality-Power Tradeoff

Learning with an Evolving Representation Language

Problem Solving Paradigm for Expert-Agent Collaboration

Knowledge Base Structuring for Knowledge Reuse

Plausible Reasoning with Partially Learned Knowledge

Integrated Teaching and Learning

Multistrategy Learning

S1

S11a

S1n

S11b1 S11bm

T11bmT11b1

T1nT11a

…

…

T1

Q1

S11bT11b

A1n S11A11

……

A11b1 A11bm

S11bQ11b

Task reduction and solution composition

guided by questions and answers

Problem Solving Paradigm for Expert-Agent Collaboration

Learning with an Evolving Representation Language

Ph.D. student

facultymember

staffmember

professor

studentuniversity employee

person

subconcept-of

subconcept-of

subconcept-ofsubconcept-of

subconcept-of

M.S. student

B.S. student

instructor

graduatestudent

undergraduatestudent

fullprofessor

associateprofessor

assistantprofessor

subconcept-of

subconcept-of

PhD_advisor

John SmithMark WhiteTom Evaninstance_of

instance_ofinstance_of

instance_ofJoan Dean

instance_of

has_as_employerGeorge Mason

University

university

.

Universe ofInstances

ExactCondition

Plausible version spaceIF<task>

THEN<subtask 1>…<subtask m>

Plausible Lower Bound Condition<PLB condition>

Plausible Upper Bound Condition<PUB condition>

Plausible Reasoning with Partially Learned Knowledge

IF<task>


Plausible Lower Bound Condition<PLB condition>

Plausible Upper Bound Condition<PUB condition>

Mixed-Initiative Problem Solving

Final RepresentationSpace

Current Representation Space

TargetSolution

Space

Correct routine solutions

Incorrectroutine

solutions

Incorrectinnovative solutions

Correct innovativesolutions

Correct inventivesolutions

Incorrectinventivesolutions

Correct creativesolutions

Final RepresentationSpace

Current Representation Space

ExpertSolutionSpace

Creative solutions

Inventive solutions

Innovative solutions

Routine solutions

Problem

Solution

Integrated Teaching and Learning

Input knowledge

Problem solving behavior

Explicit learning guidance

Explicit teaching guidance

learning hints

examples,facts,rules

classification of examples,problem solutions

questions

Rule Learning Method

Example of atask reduction

step

Plausible version space rule

analogy

PLB

PUB

Knowledge Base

Incompleteexplanation

Analogy and HintGuided Explanation

Analogy-basedGeneralization

Failureexplanation

PVSRule

Example of task reductionsgenerated by the agent

Incorrectexample

Correctexample

Learning fromExplanations

Learning by AnalogyAnd Experimentation

Learning from Examples

Knowledge BaseIF<task>


Condition<condition 1>

Except when condition<condition 2>…Except when condition<condition n>

Multistrategy Learning






Overview


Final Remarks

Challenges for the Intelligence Analyst

Overwhelmed by information

Difficult to share intelligence

A

P

Difficult to consider multiple hypotheses

Difficult to collaborate with other analysts

and experts

Difficult to avoid the analytic mindset

Difficult to analyze in reference to the culture

of the data source

Difficult to train new analysts

Difficult to find time for critical analysis

and AARsDifficult to acquire

and retain expertise

Knowledge

Difficult to rigurously explain the analysis

Intelligence analysis is

very difficult

H1 Hn

An integrated approach to intelligence analysis research, education, and operations.

Investigated Solution

Develop a new type of intelligent agent that

• can rapidly acquire expertise in intelligence analysis,

• can train new intelligence analysts, and

• can assist the analysts to solve complex problems.

Vision: Integration of Research, Education, and Operations

Agent Lifecycle

DISCIPLE-LTA

Building an agent shell

1 Knowledge engineer

2

Agent trainingby expert analyst

DISCIPLE-LTA

Expert analyst and knowledge engineer

Rapid agent development

DISCIPLE-LTA6

Knowledge baseoptimizationand re-use

Knowledge engineer and expert analyst

Agent optimization

3

DISCIPLE-LTA

Intelligent tutoring

Teaching new analysts

Analyst

Agent use andnon-disruptive learning

4

DISCIPLE-LTA

Analyst’s assistant(mixed-initiative analysis)

Analyst

After action review andagent personalization

5DISCIPLE-LTA

Analyst’s assistant(mixed-initiative learning)

Analyst

Global Grid Computing and Communications Infrastructure

JBI Platform – minimal set of standards and services

++ + +

Disciple-LTAIntelligent

agent

Disciple-LTA

Vision: Use of Disciple-LTA Agents in an Operational Environment

Disciple-LTAGLOBAL

KNOWLEDGE BASE

Disciple Client

LibrariesKnowledge Repositories

Massive Databases

SEARCH ENGINES

IntelligentAgents

Research

Mili

tary

Rese

arch

Military

Education

& PracticeDiscipleLTA

Develop a systematic approach to military intelligence analysis

Experimentation with Disciple-LTA in the 589 MAAI elective

Agent development by expert analysts using

learning agent technology

Synergistic Integration of Research and Education


Working closely with the expert analysts

in a multi-disciplinary research

Working closely with the end user

to receive crucial and

timely feedback

Live Experiment

US Army War College Course589 Military Applications of Artificial Intelligence: Intelligence Analysis

2

Agent trainingby expert analyst

After action review andagent personalization

Agent use andnon-disruptive learning

DISCIPLE-LTA

DISCIPLE-LTA

3

DISCIPLE-LTA

Intelligent tutoring

4

DISCIPLE-LTA

5

6

DISCIPLE-LTA

Knowledge baseoptimizationand re-use

Agent Lifecycle

DISCIPLE-LTA

Building an agent shell

1

Expert analyst and knowledge engineer

Knowledge engineer

Knowledge engineer and expert analyst

Rapid agent development

Teaching new analysts

Analyst’s assistant(mixed-initiative analysis)

Analyst’s assistant(mixed-initiative learning)

Agent optimization

Analyst

Analyst

Analyst

Assess whether Location-A is a training base for terrorist operationsAssess whether Location-A is a training base for terrorist operations

Assess whether Location-A is a training base for terrorist operationsAssess whether Location-A is a training base for terrorist operationsWhat type of factors should be considered to assess the presence

of a terrorist training base?

Assess whether Location-A is a training base for terrorist operations

What type of factors should be considered to assess the presence of a terrorist training base?What type of factors should be considered to assess the presence of a terrorist training base?

Political environment, physical structures, flow of suspected terrorists, weapons and weapons technology,

other suspected bases in the region, and terrorist sympathetic population


What type of factors should be considered to assess the presence of a terrorist training base?

Political environment, physical structures, flow of suspected terrorists, weapons and weapons technology, other suspected bases in the region, and terrorist sympathetic populationPolitical environment, physical structures, flow of

suspected terrorists, weapons and weapons technology, other suspected bases in the region, and terrorist

sympathetic population

Assess whether the political environment would support a training

base for terrorist operations at Location-A

Assess whether there is a flow of suspected terrorists in the region of Location-A

Assess whether there are other suspected bases for terrorist operations in the

region of Location-A

Assess whether the physical structures at Location-A

support the existence of a training base for terrorist

operations

Assess whether there are weapons and weapons technology at Location-A that

suggest the presence of a training base for terrorist operations

Assess whether there is terrorist sympathetic

population in the region of Location-A

Assess whether the political environment would support a

training base for terrorist operations at Location-A



Assess whether there is a flow of suspected

terrorists in the region of Location-A



Assess whether the physical structures at Location-A support the existence of a training base

for terrorist operations

Political environment, physical structures, flow of suspected terrorists, weapons and weapons technology, other suspected bases in the region, and terrorist sympathetic population

Assess whether there are weapons and weapons technology at Location-A that suggest the presence of a training base


Task R

edu

ction



Assess whether the political environment would support a training

base for terrorist operations at Location-A

Assess whether there is a flow of suspected terrorists in the region of Location-A



Assess whether the physical structures at Location-A

support the existence of a training base for terrorist

operations

Assess whether there are weapons and weapons technology at Location-A that

suggest the presence of a training base for terrorist operations
















Task R

edu

ction



Assess whether there are explosives at Location-A

suggesting a training base for terrorist operations

Assess whether there are chemical and/or biological agents at Location-A suggesting

a training base for terrorist operations

Assess whether there are firearms at Location-A suggesting a training

base for terrorist operations

Assess … nuclear weapons …

Assess … detonation equipment …

Assess … assembly equipment …

Assess … transportation equipment …

Assess … dispersal equipment …

Firearms, explosives, chemical and/or biological agents, nuclear weapons, transportation equipment, detonation equipment, dispersal equipment, and assembly equipment

What types of weapons and weapons technology suggest the presence of a training base for terrorist operations?














Task R

edu

ction














Assess whether there are indicators of the presence of plastic explosives at Location-A

What kind of explosives could indicate the presence of a training base for terrorist operations?

Plastic explosives because they have been used in past terrorist attacks
















Task R

edu

ction



















Assess whether there are explosive experts

in the vicinity of Location-A

What things are indicators of the presence of plastic explosives?

Assess whether there is evidence of presence of detonation equipment at

Location-A

Assess whether there is evidence of presence of secure storage facilities

at Location-A

Assess whether there is evidence of past use of plastic explosives at

Location-A

Historical evidence of past explosions, presence of explosive experts in the area, presence of detonation equipment, and presence of secure storage facilities














Task R

edu

ction























Location-A


at Location-A

Location-A contains a building with a

continuous security guard …


Location-A

Person-E, a known explosive expert, has been seen in the vicinity of

Location-A by a credible source, on several occasions

Person-X has reported a plastic explosion on

10 August 2001 at Location-A


There is a credible report that detonation equipment was brought into the country of Location-A by

truck shipment in July 2001














Task R

edu

ction























at Location-A




Location-A





Conclude that there are fairly strong indicators of the presence of plastic explosives at Location-A

So

luti

on

Syn

thes

is

These are fairly strong indicators

What kind of indicators are these for the presence of plastic

explosives at Location-A?


Location-A

















Task R

edu

ction























at Location-A




Location-A






So

luti

on

Syn

thes

is





Location-A




Conclude that there are fairly strong indicators of

the presence of plastic explosives at Location-A














Task R

edu

ction






















at Location-A




Location-A






So

luti

on

Syn

thes

is





Location-A






Conclude that there are indicators of the presence of weapons (plastic explosives, firearms) at Location-A but …















Task R

edu

ction




















at Location-A




Location-A






So

luti

on

Syn

thes

is





Location-A








Conclude that there are strong indicators that

Location-A is a training base for terrorist operations
















Task R

edu

ction




















at Location-A




Location-A






So

luti

on

Syn

thes

is





Location-A








Conclude that there are strong indicators that

Location-A is a training base for terrorist operations



Assess w hether there are explosive experts in the vicinity of ?O1

Plausible upper bound condition

Assess w hether there is evidence of presence of detonation equipm ent at ?O1

Assess w hether there is evidence of presence of secure storage facilities at ?O1

Assess w hether there are indicators of the presence of plastic explosives at ?O 1

Assess w hether there is evidence of past use of plastic explosives at ?O1

If

Plausible lower bound condition

Then

Learnedtask reduction

rule

Learnedsolution

compositionrule

There is ?O4 of presence of secure storage facilities at ?O1

There is ?O of the presence of plastic explosives at ?O1

There is ?O3 that there are explosive experts in the vicinity of ?O1

There is ?O2 of use of plastic explosives at ?O1

There is ?O5 of presence of detonation equipment at ?O1

Plausible upper bound condition

Conclude

Plausible lower bound condition

If

REVIEWER #2: This is an innovative idea that could revolutionize the way we do business, enable us to be more efficient, more effective, more thorough.

REVIEWER #1: a grand challenge to develop an intelligent agent capable of learning, tutoring and decision support …if implemented it would likely be pretty unique.

Intelligence Experts Opinion: Quotations

REVIEWER #3: a very important R&D area for next generation intelligence analysis. The work is well founded, and the execution of real software to implement the ideas is substantial.

REVIEWER #4: I have seen a briefing on the work presented here last year and was impressed with the initial ease of use of capturing complex concepts. This could be excellent for use in both training analysts as well as capturing knowledge from more senior analysts.






Overview


Final Remarks

Integrated KB

KB1

Disciple-RKF Assistant


Problem solver for a non-expert

Tutorto a student

Assistant of an expert

KBn


...Expert

Expert

Successful experiments and transition to the US Army War College

DARPA’s Rapid Knowledge Formation Program

Develop the Disciple technology to enable teams of subject matter experts to build integrated knowledge bases and agents incorporating their problem solving expertise.


If a combatant eliminates or influences the enemy’s strategic center of gravity, then the enemy will lose control of its power and resources and will eventually fall to defeat. If the combatant fails to adequately protect his own strategic center of gravity, he invites disaster.

P.K. Giles and T.P. GalvinUS Army War College, 1996.

Center of Gravity Analysis

The center of gravity of an entity is its primary source of moral or physical strength, power or resistance.

Joe Strange, Centers of Gravity & Critical Vulnerabilities,

Marine Corps War College, 1996.

0123456789

Stro

ngly

Dis

agre

e

Dis

agre

e

Neu

tral

Agr

ee

Stro

ngly

Agr

ee

0123456789

Stro

ngly

Dis

agre

e

Dis

agre

e

Neu

tral

Agr

ee

Stro

ngly

Agr

ee

0123456789

Stro

ngly

Dis

agre

e

Dis

agre

e

Neu

tral

Agr

ee

Stro

ngly

Agr

ee

589jw Military Applications of Artificial Intelligence

Students teach Disciple their COG analysis expertise,

using sample scenarios

(e.g. Iraq 2003, War on terror 2003, Arab-Israeli 1973)

Students test the trained

Disciple agent based on a

new scenario (North Korea

2003)

I think that a subject matter expert can use Disciple to build an agent, with limited assistance from a knowledge engineer

Spring 2001COG identification

Spring 2002COG identification

and testing

Spring 2003COG testing based on

critical capabilities

Global evaluations of Disciple by officers during three experiments

Use of Disciple at the US Army War College


012345678

Stro

ngly

Dis

agre

e

Dis

agre

e

Neu

tral

Agr

ee

Stro

ngly

Agr

ee

012345678

Stro

ngly

Dis

agre

e

Dis

agre

e

Neu

tral

Agr

ee

Stro

ngly

Agr

ee

DiscipleAgent KB

Problem

solving

Disciple was taught based on the expertise of Prof. Comello in center of gravity analysis.

Disciple helps the students to perform a center of gravity analysis of an assigned war scenario.

Teaching

Learning

012345678

Stro

ngly

Dis

agre

e

Dis

agre

e

Neu

tral

Agr

ee

Stro

ngly

Agr

ee

The use of Disciple is an assignment that is well suited to the course's learning objectives

Disciple should be used in future versions of this course

Use of Disciple at the US Army War College

319jw Case Studies in Center of Gravity Analysis

Disciple helped me to learn to perform a strategic COG

analysis of a scenario

Global evaluations of Disciple by officers from the Spring 05 course

Extended KB

stay informedbe irreplaceable

communicate be influential

Integrated KB

Initial KB

have supportbe protected

be driving force

432 concepts and features, 29 tasks, 18 rulesFor COG identification for leaders

37 acquired concepts andfeatures for COG testing

COG identification and testing (leaders)

Domain analysis and ontology development (KE+SME)

Parallel KB development (SME assisted by KE)

KB merging (KE)

Knowledge Engineer (KE)

All subject matter experts (SME)

DISCIPLE-COG DISCIPLE-COG DISCIPLE-COG DISCIPLE-COG DISCIPLE-COG

Training scenarios:Iraq 2003

Arab-Israeli 1973War on Terror 2003

Team 1 Team 2 Team 3 Team 4 Team 5

5 features10 tasks10 rules

Learned features, tasks, rules

14 tasks14 rules


35 tasks33 rules


Unified 2 features Deleted 4 rules Refined 12 rulesFinal KB:+9 features 478 concepts and features+105 tasks 134 tasks+95 rules 113 rules

DISCIPLE-COG

Testing scenario:North Korea 2003

Correctness = 98.15%

5h 28min average training time / team3.53 average rule learning rate / team

Parallel development and merging of KBs

KBINTEGRATION

ASSISTANT

Integration Team: Knowledge engineer +Subject matter experts

KB Integration, Validation and Maintenance

PROBLEM SOLVINGAND LEARNING

ASSISTANT

Operational Use and Non-Disruptive Learning

After Action Review and KB Refinement


ASSISTANT


ASSISTANT




ASSISTANT


ASSISTANT




ASSISTANT

Current ProjectDistributed Knowledge

Acquisition, Validation, and Maintenance

Copies of Disciple agents support users’ decision-making and all learn from these experiences.

Knowledge acquired by the agents is validated and integrated into an

improved Disciple Knowledge Base


ASSISTANT




ASSISTANT


ASSISTANT




ASSISTANT


ASSISTANT




ASSISTANT

Co-PI, SMEDr. Jerome Comello

Experiments in 2005, 2006, 2007

Co-PI, SMEDr. Joseph Strange

Experiments in 2006, 2007

Co-PI, SMECol Jeffrey HightaianLtCol Todd Kemper

Experiments in2006, 2007

Experimentation Environment

2005, 2006, 2007

Air War College

Army War College

Marine Corps War College

KBINTEGRATION

ASSISTANT

Integration Team: Knowledge engineer +Subject matter experts

KB Integration, Validation and Maintenance

George MasonUniversity






Overview


Final Remarks


Disciple-VEDisciple-VEDisciple-VEDisciple-VEDisciple-VEDisciple-VE

Virtual Experts for Multi-Domain Collaborative Planning

User’s Assistant

Virtual Team Manager

ScenarioSpecification

KB

KB

KB

KB

KB

DISTRIBUTEDKNOWLEDGE BASE

KB

Report Generator

PlanBrowser

Assistant TrainingModules

PlanAbstraction

Virtual Experts (VE) Library

Disciple-VEDisciple-VEDisciple-VE

Knowledge Management

Local Knowledge Base

Ontology Rules

Profile-based Team Selector

Disciple-VE

External-Expertise

Agent PlanGrading

Plan Brainstorming

CollaborativePlanner

VE TrainingModules

Assumption-based Reasoning

Knowledge Management

Local Knowledge Base

Ontology Rules

IndicatorsIdentification

Disciple-VE

Disciple-VE Disciple-VE

Disciple-VE Team of Virtual ExpertsDisciple-VE

User

Sample scenario: Planning the response to an emergency situation involving a tanker truck leaking red-fuming nitric acid near a student residential area of GMU.






Overview


Final Remarks

Rapid development and evaluation of a Course of Action critiquer

DARPA’s HPKB Challenge Problem

To what extent does this course of action conform to the principle of surprise?

DARPA’s HPKB Program: Evaluation

Coverage

Pe

rfo

rma

nce

0

20

40

60

80

100

120

140

160

0% 50% 100%

25% 75%

3

5

4

4

5

3

3

4 5

Coverage

(Evaluation Items 3, 4, and 5)

GMU

ISITFS/CyCorp

• High knowledge acquisition rate;

• Better results than the other evaluated systems;

• Better performance than the evaluating experts (many unanticipated solutions).

100%






Overview


Final Remarks


Disciple’s Vision on the Future of Software Development

MainframeComputers

Software systems developed and used by computer experts

PersonalComputers

Software systems developedby computer experts

and used by persons thatare not computer experts

LearningAgents

Software systems developed and used by persons that are

not computer experts

DISCIPLE

Inte

rfac

e

ProblemSolving

Learning

Ontology+ Rules


Vision on the Use of Disciple in Education


The expert/teacher teaches Disciple through examples and explanations, in a way that is similar to how the expert would teach a student.



…

Disciple tutors the student in a way that is similar to how the expert/teacher has taught it.




The research performed in the Learning Agents Center was sponsored by several US government agencies including Defense Advanced Research Projects Agency, Air Force Office of Scientific Research, Air Force Research Laboratory, National Science Foundation, and Army War College.

Acknowledgements


Questions

Documents

21 September 2005 Gheorghe Tecuci Learning Agents Center and Computer Science Department School of Information Technology and Engineering George Mason