Gheorghe Tecuci Mihai Boicu, Dorin Marcu, David Schum ...stids.c4i.gmu.edu/STIDS2011/presentations/STIDS2011_Invited_GTe… · Vision: Evolution of Software Development and Use Mainframe

Invited TalkThe Sixth International Conference on Semantic Technologies

for Intelligence, Defense, and Security – STIDSFairfax, VA, 16 November 2011

Gheorghe Tecuci

Mihai Boicu, Dorin Marcu, David Schum, Kathryn Russell

Learning Agent Center, George Mason University

2011, Learning Agents Center 2

Summary

We present the current status and applications of an evolving theoryand technology for the development of cognitive assistants by subjectmatter experts who are not knowledge engineers or computer scientists.This approach, called Disciple, synergistically integrates mixed-initiativeproblem solving, evidence-based reasoning, teaching, and multistrategylearning, enabling direct teaching of a learning agent by a subject matterexpert.The Disciple approach has been successfully employed to developcognitive assistants in a wide variety of domains, including intelligenceanalysis, modeling of violent extremists, center of gravity determination,course of action critiquing, emergency response planning, and PhDadvisor assessment.It is envisioned that this approach will enable typical computer users todevelop their own personal cognitive assistants for the semantic web.


Overview

Discussion

How a Disciple Assistant is Taught and Learns

Research and Development Objectives

Applications of the Disciple Cognitive Assistants


Alan Turing

Computing Machinery and

Intelligence

Mind, 59,433-460, 1950.

Building an intelligent machine by programming is too difficult.

“Instead of trying to produce a program to simulate the adult mind, why not rather try to produce one which simulates the child's?

If this were then subjected to an appropriate course of education one would obtain the adult brain.”

Teaching as Alternative to Programming


“Rarely does a technology arise that offers such awide range of important benefits of this magnitude.Yet as the technology moved through the phase ofearly adoption to general industry adoption, theresponse has been cautious, slow, and “linear”(rather than exponential).”

Knowledge-Based Cognitive Agents

Edward Feigenbaum

Tiger in a Cage:The Applications of Knowledge-

Based Systems

AAAI 1993Invited Talk

Building knowledge-based cognitive agents isvery difficult because much of expert’s problemsolving knowledge is in the form of tacitknowledge which is very difficult to capture.

KnowledgeEngineer

Subject MatterExpert

Knowledge Base

Inference Engine

Cognitive Agent

ProgrammingDialog

Results


The expertteaches the agent

how to solve problems in a way

that resembles how the expert would teach a

student, an apprentice or a collaborator.

The agent continuously develops and

refines its knowledge base to capture and better represent expert’s

knowledge and problem solving

strategies.

Disciple Approach to Cognitive Agent Development

Approach to develop cognitive agents that can be taught directly by subject matter experts to become cognitive assistants.

Main IdeasMixed-initiative problem solving

Evidence-based reasoningTeaching and learningMultistrategy learning

Resulting agent capabilitiesLearning expert knowledgeAssisting experts and non-experts in problem solving

Teaching students


Vision: Evolution of Software Development and Use

MainframeComputers

Software systems developed and used by computer experts

DISCIPLE

CognitiveAssistants

Software systems developed and used by

persons who are not computer experts

PersonalComputers

Software systems developed by computer

experts and used by persons who are not

computer experts


Vision: Use of Disciple Agents in Education

Personalized Learning: Grand Challenge for the 21st CenturyUS National Academy of Engineering, 2008

teachesDiscipleAgentKB

DiscipleAgentKB

Disciple behaves as a tutoring system, guiding the student through a series of lessons and exercises.

collaborate

A student uses Disciple as an assistant and learns from its

explicit reasoning.

Army Intelligence Center

teachesDiscipleAgent KB

A subject matter expert teaches Disciple similarly to how the

expert would teach a student.


Overview

Discussion




Life Cycle of a Disciple Assistant

Knowledge Base Management

MultiStrategyLearningModules

Ontology DevelopmentModules

ProblemSolvingModules

Evidence‐Specific Modules

EBR KB

Scenario KBScenario KB

Domain KBDomain KB Domain KB

Disciple Learning Agent Shell Subject Matter

Expert and Knowledge Engineer

1. ShellCustomization

Developer and Knowledge Engineer

Tutoring ModulesInteraction

Mixed Initiative Interaction

Cognitive AssistantEnd User

End Users

4. Field UseEnd User

Subject Matter Expert and Knowledge Engineer

DiscipleDisciple

Disciple

2011 Learning Agents Center

Develop ontology

Define reasoning

rules

Verify and update rulesand ontology

Very difficult and time-consuming

Model problem solving

SubjectMatter Expert

KnowledgeEngineer

S1

S11 S1

n

S21 S2

m

S31 S3p

QuestionAnswer

QuestionAnswer

QuestionAnswer

P1

P11 P1

n

P21 P2

m

P3pP31

QuestionAnswer

QuestionAnswer

QuestionAnswer

IF the problem to solve is P1

THEN solve its sub-problemsP1 … P1

Supported by minimally generalized cases

PVS Condition

Except-WhenPVS Condition

1 nJane Austin

Ph.D. student

John Doe

faculty memberstaff member

professor

studentuniversity employee

person

Bob Sharp

instance of

subconcept of

instance of instance of

subconcept of

subconcept of subconcept of

subconcept of

M.S. student

B.S. studentinstructor

graduatestudent undergraduate

student

fullprofessor

associateprofessor

assistantprofessor

subconcept of

instance of

subconcept of

Joan Dean

instance of

PhDadvisor

John Smith

graduateresearchassistant

teachingassistant

Learn reasoning

rules

Provide andexplain

examples

Analyze agent’s

solutions

Learnontologyelements

Explainerrors

Assist in problem modeling

Instruct expert to

use model

Easier and faster because many task components are performed or supported by the learning agent.

SubjectMatter Expert

KnowledgeEngineer

LearningAgent

Extend ontology based on modeled

problems

Learn /refinerules

Elicit concepts with hierarchical

organization

KE with Learning Agent Technology

Manual Knowledge Engineering (KE)

Knowledge Base Development and Maintenance Tasks

Under-stand

domain

12

Stages in Knowledge Base Development

Define types of problems to solve

Assessing a potential PhD advisor for a student

Develop ontology based on modeled problems

PhDadvisor

Jane AustinJohn Doe

faculty memberstaff member

professor

university employee


subconcept of

subconcept of

instructor

fullprofessor

associateprofessor

assistantprofessor

instance of

subconcept of

John Smith

John Doeis expert in

Artificial Intelligence

Bob Sharpis interested in

Computer Science

instance-of

Software Engineering

instance-of

research area

subconcept-of

BiologyMathematics

area of expertise

subconcept-of

Model solutions of typical problems

(rapid prototyping)

S1

S1n

QuestionAnswer

S11

S21

QuestionAnswer

S2m

P1

P11 P1

n

QuestionAnswer

P21 P2

m

QuestionAnswer

S1

S1n

QuestionAnswer

S11

S21

QuestionAnswer

S2m

P1

P11 P1

n

QuestionAnswer

P21 P2

m

QuestionAnswer

S1

S1n

QuestionAnswer

S11

S21

QuestionAnswer

S2m

P1

P11 P1

n

QuestionAnswer

P21 P2

m

QuestionAnswer

Learn rules and refine knowledge base

IF the problem to solve is P1g


Condition

Except-When Condition

1g ng

…Except-When Condition



Condition


1g ng




Condition


1g ng




Condition


1g ng




Condition


1g ng




Condition


1g ng




Condition


1g ng




Condition


1g ng


Jane Austin

Ph.D. student

John Doe

faculty member staff member

professor


person

Bob Sharp

instance of

subconcept of


subconcept of


subconcept of

M.S. student



student

fullprofessor

associateprofessor

assistantprofessor

subconcept of

instance of

subconcept of

Joan Dean

instance of

PhDadvisor

John Smith


teachingassistant

employeesubconcept of

subconcept of

S1

S1nS1

1

S21 S2

m

P1

P11 P1

n

P21 P2

mevidence-based reasoning

A problem P1 is solved by:• successively reducing it to simpler and simpler problems; • finding the solutions of the simplest problems; • successively combining these solutions to obtain the

solution of the initial problem.

General Reasoning Model: Divide and Conquer

One of the most highly developed skills incontemporary Western civilization is dissection;the split-up of problems into their smallestpossible components. We are good at it. Sogood, we often forget to put the pieces backtogether again. Alvin Toffler

S31 S3

p

S1

S1n

QuestionAnswer

S11

S21

QuestionAnswer

S2m

QuestionAnswer

P1

"I Keep Six Honest...“ by Rudyard Kipling

I keep six honest serving-men(They taught me all I knew); Their names are What and Why and When And How and Where and Who.

P11 P1

n

QuestionAnswer

P21 P2

m

QuestionAnswer

P3pP3

1

QuestionAnswer

Problem Reduction and Solution Synthesis

Assess whether John Doe would be a good PhD advisor for Bob Sharp.

PhD Advisor Assessment: Overall Logic

It is almost certain that John Doe would be a good PhD advisor with respect to the

professional reputation criterion.

It is very likewould be a gwith respect

student re

It is verwould b

with advis

It is almost certain that John Doe will stay on the faculty of George Mason University for the duration

of dissertation of Bob Sharp.

It is certain that Bob Sharp is interested in the area of expertise

of John Doe.

It is very likely that John Doe would be a good PhD advisor for Bob Sharp.

Assess whether John Doe will stay on the faculty of George

Mason University for the duration of dissertation of Bob Sharp.

Assess whether John Doe would be a good PhD advisor

with respect to the PhD advisor quality criterion.

Assess whether Bob Sharp is interested in the area of expertise

of John Doe.

Which are the necessary conditions?Bob Sharp should be interested in the area of expertise of John Doe who

should stay on the faculty of George Mason University for the duration of the dissertation of Bob Sharp, and should have the qualities of a good PhD advisor.

Assess whether John Doewould be a good PhD advisor

with respect to the professional reputation criterion.

…

Assess whether John Doewould be a good PhD advisor with respect to the quality of

student results criterion.

Which is a PhD advisor quality criterion?

professional reputation criterion

Which is a PhD advisor quality criterion?quality of student results criterion

Yves Kodratoff

PhD Advisor

Hybrid Knowledge Base = Ontology + RulesOntology: Hierarchical representation of both general and domain-

specific conceptsevidence

tangible evidence

testimonial evidence

demonstrative tangible evidence

real tangible evidence

unequivocal testimonial evidence

equivocal testimonial evidence


based upon direct

observation

authoritative record

missing evidence


obtained at second hand

testimonial evidence based on opinion

completely equivocal


probabilistically equivocal


Problem reduction and solution synthesis rules: Specified with the concepts

from the ontology


Condition


1g ng



Condition


1g ng



Condition


1g ng




Condition


1g ng




Condition


1g ng




Condition


1g ng




Condition


1g ng




Main Condition


1g ng


Main Condition Except-WhenCondition

Except-WhenCondition

Reasoning tree:Finding the solution of a

specific problem

S1

S1 S1

S2 S2P2P2

P1P1

P1

…

…

S3 S3P3P3 …

1 1 n n

1 1 m m

1 1 p p

2011 Learning Agents CenterJane Austin

Ph.D. student

John Doe

faculty member staff member

professor


person

Bob Sharp

instance of

subconcept of


subconcept of


subconcept of

M.S. student



student

fullprofessor

associateprofessor

assistantprofessor

subconcept of

instance of

subconcept of

Joan Dean

instance of

PhDadvisor

John Smith


teachingassistant

employeesubconcept of

subconcept of

Learned with an evolving ontology

16

Partially Learned Rules and Evolving Ontology

IF the problem to solve is P1


PVS Condition


1 n

S1

S11 S1

n

S21 S2

m

S31 S3p

QuestionAnswer

QuestionAnswer

QuestionAnswer

P1

P11 P1

n

P21 P2

m

P3pP31

QuestionAnswer

QuestionAnswer

QuestionAnswer

Partially learned

rules with plausible version space (PVS)

conditions

+

-

PVS Condition Except-WhenPVS Condition

Reasoning Tree

Mixed-Initiative Problem Solving

AcceptReasoning Step

RejectReasoning Steps

Rules Refinement

Problem

ExtendReasoning Tree

ExplainExamples

Rules Learning

ExplainExamples

ExplainExamples

Refined Rules

Refined Ontology

Learned Rules

Modeling, Learning, and Problem Solving

Ontology + RulesS3

1 S3p

S11

S21

QuestionAnswer

S2m

QuestionAnswer

P1

P11 P1

n

QuestionAnswer

P21 P2

m

QuestionAnswer

P3pP31

QuestionAnswer

18

The subject matter expert helps the agent to learn (e.g. byproviding examples, hints, and explanations), and the agenthelps the expert to teach it (e.g. by presenting attemptedsolutions to problems and by asking relevant questions).

Integrated Teaching and Learning

Input knowledge

Problem solving behavior

Explicit learning guidance

Explicit teaching guidance

Explanations, hints and answers to

agent’s questions

Problem solving examples

Attempted solutions to problems

Questions

P1

P11 P1

n

Question QAnswer A

…

Example

Rule Learning Method

Knowledge Base

1. Ontology-based mixed-initiative understanding

f1

ob1 ob2

ob3

f2

Explanation

Knowledge Base

3. Minimal and maximal ontology-based generalizations guided by analogical reasoning



1g ng

QgAg

Applicability condition

2. Example and explanation

reformulation



1g ng

QgAg

?O1 is ob1f1 ?O3

?O2 is ob2f2 ?O3

?O3 is ob3

20

The expert makes explicit how to solve a

problem

1. Modeling

Modeling and Learning

LearnedRule2

LearnedRule1

The agent learns reduction rules

2. Learning

The expert and the agent

identify formal

explanations of the

reductions

Understand Why the Example Is Correct

Approximate representation of the

meaning of the question and its

answer in the ontology.

John Doe

Artificial Intelligenceis expert inis interested in

Bob Sharp

Likelihood of solution is always certain

An even better formulation of the answer is:Yes, this is certain because Bob Sharp is

interested in Artificial Intelligence which is the area of expertise of John Doe.

John Doe

PhD advisor

Bob Sharp


PhD student

employee

subconcept of

person

associate professor

subconcept of

graduate student

student

subconcept of

professor

faculty member

university employee

actor

object

Artificial Intelligence

Computer Science

area of expertise

instance of

feature

domain

range

personis interested in

subfeature of

area of expertise

Object ontology

subconcept of

?O1 is Bob Sharpis interested in?O3

?O2 is John Doeis expert in ?O3

?O3 is Artificial Intelligence?SI1 is-exactly certain?O3

?O1?O2

Feature ontology

Min - Max Generalizations



1g ng

QgAg

?O1 is ob1f1 ?O3

?O2 is ob2f2 ?O3

?O3 is ob3

UB

UB

?O1 is personis interested in?O3

?O2 is personis expert in ?O3

?O3 is area of expertise?SI1 is-exactly certain

Plausible UpperBound Condition (UB)

Max generalization

LB

LB

LB

LB

?O1 is PhD studentis interested in?O3

?O2 is {associate professor, PhD student}

is expert in ?O3?O3 is Artificial Intelligence?SI1 is-exactly certain

Plausible LowerBound Condition (LB)

Min generalization

Positive example

Explanation generalizations

Learned rulePartially learned applicability

condition and rule

Pattern generation

24

Agent applies learned rules to

solve new problems

3. Solving

Expert accepts or

rejectsindividualreductions

4. Critiquing

Solving, Critiquing, and Refining

Agent refines learned rules with the new positive and

negative examples

5. Refining

RefinedRule1

RefinedRule2

25

Failureexplanation

Examples of problem reductions generated by agent

Incorrectexample

Learning fromExplanations

Learning by Analogyand Experimentation

Learning from Examples

Knowledge Base

Refinement Strategy: PVS Condition Refinement

_++

Tom Mitchell

Version spaces: A candidate elimination approach to rule learning

IJCAI 1977

OptionalExplanation

Correctexample

Rule’s main condition

?O1 is Bob Sharpis interested in ?O3

?O2 is Dan Smithis expert in ?O3

?O3 is Information Security?SI1 is-exactly certain

Condition corresponding to the example

26

Positive example

Rule Generalization

Generalized lower bound

27

Solving, Critiquing, and Refinement

Agent applies learned rules to solve new problems1. Solving

2. Critiquing

Incorrect because

Dan Smith plans to

retire

3. Refinement

Agent refines rule with negative example

+

+-

28

Failureexplanation

Example of problem reductionsgenerated by the agent

Incorrectexample

Correctexample

Learning fromExplanations

Learning by Analogyand Experimentation

Learning from Examples

Knowledge Base

IF we have to solve<Problem>

THEN solve<Sub-problem 1>…<Sub-problem m>

MainPVS Condition


Refinement Strategy: Except-When PVS Condition

+-

29

Rule SpecializationNegative example

Rewrite as

?O1 is Dan Smithplans to retire from ?O2

?O2 is George Mason University

Except when condition

Refined rule

+-

Dan Smith plans to retire from George Mason University

Failure explanation

30

Solving, Critiquing, and Refinement

Agent applies learned rules to solve new problems1. Solving

2. Critiquing

Incorrect because

Jane Austin

plans to move

3. Refinement


+-

+ --

31

Rewrite as

?O1 is Jane Austinplans to move to ?O6

?O5 is Indiana University

Except when condition

Refined rule

Jane Austin plans to move to Indiana UniversityFailure explanation

Negative example

+ --

32

Agent applies learned rules to

solve new problems

1. Solving

Solving, Modeling, and Learning

2. Modeling

Expert extends the reasoning

tree

Agent learns a new rule

3. Learning

+

33

Rule Learning

Example andexplanation

Learned rule

Partially learned applicability condition

and rule

34

Solving, Critiquing, and RefinementAgent applies learned rules to solve new problems1. Solving

2. Critiquing

Incorrect because of the “even chance”

likelihood

3. Refinement


+

+ -

35

Explain Why the Problem Reduction Is Incorrect

Failure explanation proposed by

the agent

Negative example

Refined rule

36

Refined RuleNegative example

Specialized upper bound

Failure explanation

Main condition

37

ProblemSolving

Modeling

LearningExpert

example

Rule-basedguidance

Creativesolution

Context forcreative solution

Refinedrule

Newexamples

Synergy of Modeling, Learning, and Problem Solving

Modeling, learning, and problem solving mutually

support each other to capture tacit knowledge

Mixed-Initiative

Reasoning

38

Features of the Disciple Learning Method

Uses multistrategy learning (examples, explanations, analogy)

Learns rapidly from a small number of examples

Learns even in the presence of exceptions

Captures expert’s tacit knowledge

Is applicable to complex real-world domains

+

-


Overview

Discussion




40

Course of Action Critiquing

Workaround Reasoning


Center of Gravity Determination

Web Believability Evaluation

Modeling of Violent Extremists

Regulatory Compliance in Financial Services Industry

Emergency Response Planning

PhD Advisor Assessment

Intelligence Analysis

Capturing of Learned Lesson

Detailed reasoning for selected problemAbstract reasoning tree

Application of the captured

lesson to a new situation

42


Questions

This research was performed in the Learning Agents Center and wassupported by George Mason University and by several agencies ofthe U.S. Government, including the Department of Defense, theNational Geospatial-Intelligence Agency, the Intelligence Community,the Air Force Office of Scientific Research, the Air Force ResearchLaboratory, the Defense Advanced Research Projects Agency, theNational Science Foundation, the U.S. Army War College, and theJoint Forces Staff College.

The views and conclusions contained in this document are those ofthe authors and should not be interpreted as necessarilyrepresenting the official policies or endorsements, either expressedor implied, of the U.S. Government.

The U.S. Government is authorized to reproduce and distributereprints for Government purposes notwithstanding any copyrightnotation thereon.

Acknowledgements

Documents

Gheorghe Tecuci Mihai Boicu, Dorin Marcu, David Schum ...stids.c4i.gmu.edu/STIDS2011/presentations/STIDS2011_Invited_GTe… · Vision: Evolution of Software Development and Use Mainframe