Lesson 2: The JDL Model David L. Hall. Lesson Objectives Introduce the Joint Directors of Laboratories (JDL) Data Fusion Process Model Compare the JDL

Lesson 2: The JDL Model

David L. Hall

Lesson Objectives

• Introduce the Joint Directors of Laboratories (JDL) Data Fusion Process Model

• Compare the JDL Model with competing models

Introduction and Motivation• The JDL Data Fusion Process Model has

proliferated throughout the DoD fusion community– Original creation in 1991/1992– Multiple extensions/modifications– Proliferation in books, papers, conferences– Utilized in organizations, RFPs, etc.

• Emerging models from other community models and applications– Decision support– Cognitive sciences

Overview of Models Surveyed

Boyd (1976)Boyd (1996)

A process model of military decision makingOODA

Klein (1997)Klein (1999)

Decision-Making

A naturalistic theory of decision making focused on recognition of perceptual cues and action

Recognition Primed Decision Making

Bedworth and O’Brien (1999)Adaptation of Boyd’s OODA loop for data fusion

Omnibus Model

Kessler and Fabien (2001)Application of the waterfall development process to data fusion

TRIP Model

Dasarathy (1994)An abstraction of input-output functions of the data fusion process

Functional Levels of Fusion

Hall and Llinas (1997)Hall and Llinas (2001)Hall and McMullen (2004)Steinberg (1998)

Data FusionA functional model for describing the data fusion process

JDL

ReferencesFieldDescriptionModel

Summary of reviewed process models

Boyd (1976)Boyd (1996)

A process model of military decision makingOODA

Klein (1997)Klein (1999)

Decision-Making

A naturalistic theory of decision making focused on recognition of perceptual cues and action

Recognition Primed Decision Making

Bedworth and O’Brien (1999)Adaptation of Boyd’s OODA loop for data fusion

Omnibus Model

Kessler and Fabien (2001)Application of the waterfall development process to data fusion

TRIP Model

Dasarathy (1994)An abstraction of input-output functions of the data fusion process

Functional Levels of Fusion

Hall and Llinas (1997)Hall and Llinas (2001)Hall and McMullen (2004)Steinberg (1998)

Data FusionA functional model for describing the data fusion process

JDL

ReferencesFieldDescriptionModel

Summary of reviewed process models

Origin of the JDL Data Fusion Model

• JDL Data Fusion Sub-Panel (and working group)

• Meeting in State College, PA

• Development of briefing for the Office of Naval Intelligence

Members of the Joint Directors of Laboratories (JDL) Data Fusion Working Group: Ed Waltz, Chee Chong, Frank White, Otto Kessler, David Hall, James Llinas and Alan Steinberg

JDL Data Fusion Model

Top level view of the JDL data fusion process model (Hall and McMullen (2004))

Example of 2nd Level in JDL Model Hierarchy

JDL Level One Processing Object Refinement

Data Alignment

• Spatial Reference Adjustment• Temporal Reference Adjustment• Units Adjustment

Data/Object Correlation

Object Positional & Kinematic

Attribute Estimation

• System Models• Optimization Criteria• Optimization Approach• Processing Approach

Object Identity Estimation

• Physical Models• Feature-based Inference Techniques• Cognitive-based Models

CA

TE

GO

RY

FU

NC

TIO

NP

RO

CE

SS

• Gating• Association Measures• Assignment Strategies

SourcesHuman

ComputerInteraction

DATA FUSION DOMAIN

Level OSignal

Refinement

Level OneObject

RefinementLevel TwoSituation

Refinement

Level ThreeThreat

Refinement

Level FourProcess

Refinement

Database Management System

SupportDatabase

FusionDatabase

JDL Level One Processing Object RefinementJDL Level One Processing Object Refinement

Data Alignment

Data Alignment



Data/Object Correlation





Object Identity Estimation



CA

TE

GO

RY

FU

NC

TIO

NP

RO

CE

SS

CA

TE

GO

RY

FU

NC

TIO

NP

RO

CE

SS



SourcesHuman

ComputerInteraction

DATA FUSION DOMAIN

Level OSignal

Refinement

Level OneObject


Refinement

Level ThreeThreat

Refinement

Level FourProcess

Refinement


SupportDatabase

FusionDatabase

SourcesHuman

ComputerInteraction

DATA FUSION DOMAIN

Level OSignal

Refinement

Level OneObject


Refinement

Level ThreeThreat

Refinement

Level FourProcess

Refinement


SupportDatabase

FusionDatabase

Example of 3rd Layer in JDL Model Hierarchy

Object Positional & Kinematic Attribute Estimation

CA

TE

GO

RY

FU

NC

TIO

NT

EC

HN

IQU

E

System Models

Optimization Criteria

Optimization Approach

Processing Approach

• Observation Equations• Equations of Motion• Dynamic Maneuver Model• State Vector Definition• Implementation

- Data Editing- Coordinate Systems

• Direct Methods- Non-derivative Methods- Downhill Simplex- Direction Set- Derivative Methods- Conjugate Gradient- Variable Metric (Quasi-

Newton)• Indirect Methods

- Newton-Raphson Methods

• Sequential Processing- Kalman Filter- Filter

• Batch Processing• Covariance Error

Formulation

• Least Squares (LS)• Weighted LS• Mean Square Error• Maximum Likelihood• Constrained (Bayesian)


CA

TE

GO

RY

FU

NC

TIO

NP

RO

CE

SS




CA

TE

GO

RY

FU

NC

TIO

NT

EC

HN

IQU

E

System Models



Processing Approach








Formulation



CA

TE

GO

RY

FU

NC

TIO

NT

EC

HN

IQU

EC

AT

EG

OR

YF

UN

CT

ION

TE

CH

NIQ

UE

System Models



Processing Approach

System ModelsSystem Models





Processing Approach

Processing Approach








Formulation



CA

TE

GO

RY

FU

NC

TIO

NP

RO

CE

SS




CA

TE

GO

RY

FU

NC

TIO

NP

RO

CE

SS



Transformation of Requirements for the Information Process (TRIP) Model

Sensing

Signal Processing

Feature Extraction

Pattern Processing

SituationAssessment

DecisionMaking

Sensing

Signal Processing

Feature Extraction

Pattern Processing

SituationAssessment

DecisionMaking

The TRIP waterfall model for data fusion system development (adapted from Kessler and Fabien (2001))

Omnibus Model

Signal Processing

Sensing

Observe

Pattern Processing

Feature Extraction

Orien

t

Decision Making

Context Processing

Decide

Control

Resource Tasking

Act

Sensor Management

Hard Decision Fusion

Soft Decision Fusion

Sensor Data Fusion

Feature Fusion

The Omnibus Model for decision making and data fusion (adapted from Bedworth and Obrien (1999))

Dasarathy’s Functional Model

Three general levels of abstraction in fusion processing: Data level - integration of raw observations and can occur only in the case when the observations are of the same type Feature level - assumes that each stream of sensory data is first analyzed for features, after which the features themselves are fused Decision Level - based on the fusion of individual mode decisions or interpretations

Table 2: Components of Dasarathy’s Model

Input Output Notation Analogues

Data Data DAI-DAO Data-level fusion

Data Features DAI-FEO Feature selection and feature extraction

Features Features FEI-FEO Feature-level fusion

Features Decisions FEI-DEO Pattern recognition and pattern processing

Decisions Decisions DEI-DEO Decision-level fusion

Recognition Primed Decision-Making

Recognition Primed Decision Making under dynamic evolving situations (adapted from Klein (1999)).

Experience the Situation in a Changing

Context

• Reassess Situation• Seek More Information

Mental Simulation of Action

Modify

Implement

Activation of Information from Memory:

• Plausible Goals• Expectancies• Relevant Cues• Actions

Are expectancies

violated?

Is the Situation Familiar?

Will it work?

No

YesYes

No

Yes

Yes

No

Experience the Situation in a Changing

Context

• Reassess Situation• Seek More Information

Mental Simulation of Action

Modify

Implement





Are expectancies

violated?

Is the Situation Familiar?

Will it work?

No

YesYes

No

Yes

Yes

No

Intelligent Agents to Support Team Cognition

Motivation and Vision• Effective human teams use shared

mental models (SMM) to anticipate & satisfy the needs of teammates.

• Our vision: Empower software agents with a cognitively inspired SMM to support human decision makers to overcome information overload under time stress.

R-CAST Implementation• Inspired by Recognition-Primed

Decision Model (RPD).• Integrates information seeking in a

collaborative decision-making process• Supports context-centric information

sharing.• Enables collaborative/automated

reasoning.

Shared MentalModel

Information Fusion 2+

Information Fusion 1



Team DecisionContext

Computational SMMContext

Shared MentalModel













A cyber-advisory team of intelligent agents supports

collaborative decision making andmitigate cognitive biases.

http://ist.psu.edu/yen/Lab/

Boyd’s OODA Loop

Observe

Orient

Decide

Act

The original OODA loop (adapted from Boyd (1987))

Cover of Boyd’s biography: Downloaded from www.amazon.com web site on July 27, 2008

http://www.amazon.com/

Boyd’s Updated OODA Loop (1996)

Observe Orient Decide Act

Observations

Cultural Traditions

Genetic Heritag

e

New Information

Previous Experience

Analysis & Synthesis

Decision Hypothesis

Unfolding Interaction

with Environment

Unfolding Interaction

with Environment

Implicit Guidance &

Control

Implicit Guidance &

Control

Feedback

Feedback

Action

Boyd’s modern OODA Loop decision process model (1996)

OODA Variations

Variations of OODA Loop ModelModel Description References

CECA An modified version of OODA using modern theories of cognition

Bryant, 2005

M-OODA Modular OODA adds process, state, & control components

Rousseau & Breton, 2004

T-OODA Team OODA is a modified M-OODA scoped for team decision making


C-OODA Cognitive OODA includes the theories of RPD and SA


D-OODA Dynamic OODA adds planning and Sensemaking with a focus to increase the speed of decision-making

Brehmer, 2005

SADT OODA A functional system design representation of OODA

Grant, 2005

Evaluating the Scope of Models

Sensing & Data Gathering

Data Fusion Core Functions

Human Computer Interaction

Situation Assessment

Decision Making

Action & Feedback

JDL

TRIP

Dasarathy

Omnibus

RPD

OODA

OODA Variations

Strong Coverage

Weak Coverage

Key:

Endsley’s Model of Situational Awareness

Downloaded on July 29, 2008 from http://en.wikipedia.org/wiki/Situational_awareness

http://en.wikipedia.org/wiki/Situational_awareness

Summary

• JDL Fusion model has evolved to remain useful

• Extensions to level’s 0 and level 5 increase the utility and coverage of the model

• Future extensions– More attention to decision-making– Incorporation of cognitive models– Maturation of 2nd & 3rd level hierarchy for levels 2 and 3– Blurring of levels– Explicit understanding of distributed processing effects– Extension of the concepts of data and information

Lesson 2 Assignments• Preview the on-line lesson 2 materials• Read Chapter 2 of Hall and McMullen• Read Hall, Hellar, Llinas & McNeese 2007 paper• Writing assignment 3: Using the example

introduced in the first session, write a one page paper describing how the JDL model represents components of that process, e.g., what are the sensing and diagnosis functions that relate to levels 1, 2, 3, 4 and 5. What are the sensor inputs? What are the human inputs? What inferences or decisions are trying to be made? Why is it necessary to use multiple sensors?

Data Fusion Tip of the Week

Tip: “It helps to remember that there is no spoon” from the movie, The Matrix, March 1999Data Fusion Corollary: “It helps to remember, there are no levels of data fusion”

Documents

Lesson 2: The JDL Model David L. Hall. Lesson Objectives Introduce the Joint Directors of Laboratories (JDL) Data Fusion Process Model Compare the JDL