1 “Experimental Study of Automation to Support Time-Critical Replanning Decisions” Kip E. Johnson, MIT Dept. of Aero & Astro Engineering Advisors Jim Kuchar

MIT ICATMIT ICAT

1

“Experimental Study of Automation to Support Time-Critical

Replanning Decisions”

Kip E. Johnson, MIT Dept. of Aero & Astro Engineering

Advisors Jim Kuchar & Chuck Oman

Sponsored by Office of Naval Research

JUP Conference, 21-22 June 2001Hosted by Ohio University

Sponsored by NASA & FAA

MIT ICATMIT ICAT

2

“Experimental Study of Automation to Support Time-Critical Replanning Decisions”

OVERVIEW:

Problem Identification

Experimental Design

MIT ICATMIT ICAT

3

MOTIVATION: Decision-Making in Complex Environments

Replanning Under Time Pressures

PROBLEM UNIQUENESS: Automation will Never “See” Everything

Difficulty in Quantifying a Solution’s ValueUnstructured Aspects to Problem

Multiple Competing Interests & Goals for Solution


MIT ICATMIT ICAT

4

Sensors DisplaysHuman

Actuator

SensorsAutomation

Actuator

Environment

Pro

cess

Information Transduction

Decision Making

Control / Actuation

Interface


Human-Automation Interaction in a Decision-Support Task

Adopted from J. Kuchar

MIT ICATMIT ICAT

5


Example Applications:

• Military Combat• Aviation• Medicine• Chemical and Energy Production• Finances

Focus on “Real-Time In-Flight Replanning”Route Replanning

MIT ICATMIT ICAT

6


Replanning Task Characteristics

Threats/Hazards

Time on Target

Fuel Constraint

Information Integration

Solution

Information Elements

H A

MIT ICATMIT ICAT

7

QUESTION?

To What Degree should the Replanner Automation Filter and Integrate Information?• Take Advantage of Human’s Intuition and Ability to Integrate Diverse and

Complex Information.• Replanner Should Have “SMART” Automation Logic Based on Task and Time

Pressures.

HYPOTHESIS:

As Time Pressure Decreases, More Integrated Automation Support May Hinder Pilot Performance.


MIT ICATMIT ICAT

8


Task Timescale

Lev

el o

f In

form

atio

n I

nte

gra

tio

n i

n A

uto

mat

ion

Subsecond Second Minute Hour

Low

Hig

h

InformationOverload

Good Observability

Time Available to AssimilateSupport with Other Info

Opaque SupportCannot Assimilate Automated

Guidance w/ Other Info

Timely, Specific Solution(Must Use Correct Info)

Generalized System Design

Notional Hypothesis of Interaction Between Information Integration & Task Timescale

MIT ICATMIT ICAT

9

RESEARCH GOALS:

1. Find a Quantifiable Relationship Between:• Time Pressures• Degree of Information Integration in Automation• Resulting Decision Performance

2. Build a Generalized Model of Decision Support Automation.

3. Identify Information Support Needs of Human


MIT ICATMIT ICAT

10


EXPERIMENTAL DESIGN:

Replanning Protocol:

• View Preplanned Mission

• Change in Environment

• Hazard, Time on Target, or Fuel Update

• Route Suggested with Varying Levels of Information Integration

• Subject Modifies Flight Plan Under Time Pressure

• Minimize Threat Exposure and Time on Target Deviation

• Meet Time on Target and Fuel Constraints

MIT ICATMIT ICAT

11


Sequence of Events

Time PressureViewScene

Unlimited Replanning

Auto-RouteSuggestion

Mission Change:Hazard,ToT,Fuel

Time

Replanning

Performance Data Recorded

MIT ICATMIT ICAT

12


Pre-Planned Mission

Constraint Gauges

Original Route

Hazards

Start

Target

Must-Fly

Finish

MIT ICATMIT ICAT

13


Updated Mission Plan

Suggested Route

Constraint Gauges

Alerts

Time Pressure

MIT ICATMIT ICAT

14


COST FUNCTION:

CostRoute = CostHazard + CostToT

Fuel = Constraint

1*exp**0

11 t

tbaBCostLengthACost

ColorsColorntRouteSegmeRoute

MIT ICATMIT ICAT

15

Route Total Cost vs. Time

15000

17000

19000

21000

23000

25000

27000

29000

31000

33000

0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0

Time (sec)

Co

st

0

1

Inherent Route Cost

Time PressuredRoute Difference

No Pressure Route Difference


Dependent Variable:

Time Pressure

MIT ICATMIT ICAT

16

INDEPENDENT VARIABLES:

Time Pressures (TBD):


Area of Interest

Time Pressure

Per

form

ance T1 T2 T3

MIT ICATMIT ICAT

17

INDEPENDENT VARIABLES:

Information Elements Integrated by Automation

1. No Automation, Manual Replan Original Route Remains

2. Constraint Information Filtration Only Route Modified to Optimize Time on Target Deviation &

Satisfice Fuel Constraint

3. Threat Information Filtration Only Route Modified to Avoid/Minimize Hazard Levels

4. Integration of Constraint + Threat Field Information Route Minimizes Threat Exposure + Satifices Time on

Target and Fuel Constraints


MIT ICATMIT ICAT

18


Optimize ToT, Meet FuelConstraint

MIT ICATMIT ICAT

19


Minimize Threat Exposure

MIT ICATMIT ICAT

20


Satisfice Constraints +

Minimize Threat Exposure

MIT ICATMIT ICAT

21


Experimental Test MatrixInformation Automation

1 2 3 4

a

b

c

Tim

e P

res

su

res

3 by 4 Test Matrix

• Counterbalanced• Scenarios of Similar Complexity• Repeated-Measures Analysis of Variance

MIT ICATMIT ICAT

22


Status:

In-Flight Replanner Software Developed

Generating Scenarios for Pilot Experiment

Future: Run Data Collection w/ Subjects

Refine Pilot Experiment for Formal Study

MIT ICATMIT ICAT

23


Documents

1 “Experimental Study of Automation to Support Time-Critical Replanning Decisions” Kip E. Johnson, MIT Dept. of Aero & Astro Engineering Advisors Jim Kuchar