1

ISLE Transfer Learning TeamMain Technology Components

The ICARUS Architecture

Markov Logic Networks

Executes skills in the environment

Long-TermLong-TermConceptualConceptual

MemoryMemory

Short-TermShort-TermConceptualConceptual

MemoryMemory

Short-TermShort-TermGoal/SkillGoal/SkillMemoryMemory

ConceptualConceptualInferenceInference

SkillSkillExecutionExecution

PerceptionPerception

EnvironmentEnvironment

PerceptualPerceptualBufferBuffer

Problem SolvingProblem SolvingSkill LearningSkill Learning

MotorMotorBufferBuffer

Skill RetrievalSkill Retrieval

Long-TermLong-TermSkill MemorySkill Memory

Contains relational and hierarchical knowledge

about relevant concepts

Generates beliefs using observed environment and long term

conceptual knowledge

Creates internal description of the

perceived environment

Contains descriptions of the perceived objects

Contains inferred beliefs about the environment

Contains hierarchical knowledge about executable skills

Finds novel solutions for

achieving goals

Acquires skills from successful problem

solving traces

Selects relevant skills based on beliefs and goals

Contains goals and intentions

The Soar Architecture

The Companions Architecture

Body

Long-Term MemoriesProcedural

Short-Term Memory

Dec

isio

n P

roce

dure

Chunking

Episodic

EpisodicLearning

SemanticLearning

Semantic

ReinforcementLearning

Perception Action

MarkovLogic

WeightedSatisfiability

Markov ChainMonte Carlo

Inductive LogicProgramming

WeightLearning

TargetDomain

SourceDomain

2

Year 1: Transfer in Three Testbeds

1. A ball is released from rest from the top of a 200 m tall building on Earth and falls to the ground. If air resistance is negligible, which of the following is most nearly equal to the distance the ball falls during the first 4 s after it is released?

(A) 20 m (gt/2) (B) 40 m (gt)

(C) 80 m ( 2 2gt )

(D) 160 m ( 2gt ) Concepts: One dimensional kinematics with constant acceleration, distance-time relationship.

Rationale: From the distance-time relationship for constant acceleration, 20 2x v t at= + , initial

speed 0v is zero so the distance is equal to 2 2gt , where g, the acceleration due to gravity, is

approximately 10 2m s .

Urban Combat is a first-person shooter game that involves spatial reasoning and reactive control

General Game Playing covers a broad class of N-person games that involve strategic reasoning.

ETS Physics involves finding answers to physics problems through a mixture of plausible inference and search

Crawl under

Climb over

UrbanCombat

ETSPhysics

GGP

Soar

Companions

ICARUS

Our Year 1 efforts focused on Urban Combat (Soar and ICARUS), but with some work on GGP (ICARUS) and ETS Physics Companions)

source source

targettarget

Year 1 emphasizes lower levels (1 to 8) of transfer learning

3

ISLE Team: Year 2 Integration Plans

Alchemy (Washington)

Long-TermLong-TermConceptualConceptual

MemoryMemory

Short-TermShort-TermConceptualConceptual

MemoryMemory

Short-TermShort-TermGoal/SkillGoal/SkillMemoryMemory

ConceptualConceptualInferenceInference

SkillSkillExecutionExecution

PerceptionPerception

EnvironmentEnvironment

PerceptualPerceptualBufferBuffer

Problem SolvingProblem SolvingSkill LearningSkill Learning

MotorMotorBufferBuffer

Skill RetrievalSkill Retrieval

Long-TermLong-TermSkill MemorySkill Memory

Reinforcement Learning (UT Austin)

WeightedSatisfiability

Markov ChainMonte Carlo

Inductive LogicProgramming

WeightLearning

MarkovLogic

β(A’)→A

γ(S’)→S

1

2:S A ' :S' A '

ICARUS (ISLE)

Hierarchical Task Networks (Maryland)

InsertCycorp

Mountainhere

CYC (Cyrcorp)

Year 2 integration will revolve around replacing existing ICARUS modules with software from other team members

4

Year 2: Deep Transfer in Three Testbeds

In Urban Combat, we will demonstrate transfer from urban military missions to fire rescue operations

In General Game Playing, we will show transfer across quite different games that use related strategies(e.g., forking moves)Crawl under

Climb over

UrbanCombat

ETSPhysics

GGP

Soar

Companions

ICARUS

In Year 2, We will evaluate each pair of architectures on at least one shared testbed and we will test them all on the General Game Playing testbed

source

source

targettarget

Year 2 will focus on higher levels (9 -10) of transfer learning

In ETS Physics, we will show transfer from linear systems to rotational, thermal, hydraulic, and electrical systems

source

target

5

ISLE Team: Year 2 Evaluation Plans

Comparison among architectures that use different mechanisms should reveal which approaches best support transfer learning.

We will evaluate each pair of architectures on at least one shared testbed and will test them all on the General Game Playing testbed.

Experiments will examine how well the frameworks support transfer in settings that emphasize reactive control, conceptual inference, and heuristic search.

Year 2 evaluations will focus on high-level (9 and 10) transfer in each of the three testbeds.

UrbanCombat

ETSPhysics

GGP

Soar

Companions

ICARUS

Arc

hite

ctur

es

Testbeds