Team Training Paradigm forTeam Training Paradigm forbetter Combat Identificationbetter Combat Identification

Wayne ShebilskeWayne Shebilske•• (Wright State University)(Wright State University)

Jared Freeman,Jared Freeman, GeorgiyGeorgiy LevchukLevchuk, &, & Kevin Kevin GildeaGildea,,

•• ((AptimaAptima))

Combat Identification in the AirCombat Identification in the AirForce Offensive TeamForce Offensive Team

RecognizeRecognize•• What an object of interest isWhat an object of interest is•• Whether it is expectedWhether it is expected•• Whether it is a threat or opportunityWhether it is a threat or opportunity•• Whether it demands a response within theWhether it demands a response within the

current mission as specified by the Air Taskingcurrent mission as specified by the Air TaskingOrder (ATO)Order (ATO)

•• Whether available friendly assets have theWhether available friendly assets have thecapacity to respond.capacity to respond.

These multiple dimensions are interacting,These multiple dimensions are interacting,interdependent components of a complexinterdependent components of a complexwhole combat identification system.whole combat identification system.

Training Experienced Teams forTraining Experienced Teams forNew ExperiencesNew Experiences

Laboratory SimulationLaboratory Simulation•• Focus on DynamicFocus on Dynamic

Targeting Cell (DTC)Targeting Cell (DTC)•• Combined AerospaceCombined Aerospace

Operation CenterOperation Center(CAOC)(CAOC)

•• AptimaAptima’’ss DDD DDD Synthetic TaskSynthetic Task

Environment (STE)Environment (STE)

IV: Interventions to aidIV: Interventions to aidTraining of ExperiencedTraining of ExperiencedTroopsTroops

DV: BenchmarkedDV: BenchmarkedExperiential Systems forExperiential Systems forTraining (BEST)Training (BEST)•• ExpertsExperts•• Intelligent AgentsIntelligent Agents•• Mathematical ModelsMathematical Models

CognitiveScience

(e.g., info processing capacity)

Management&

OrganizationalScience

(e.g., centralized vs.decentralized)

MilitaryScience

(e.g. principles of war)

DynamicTargetingProblem

Suggests dimensions,

parameters and hypothesis.

Suggests dimensions,parameters and hypothesis.

Suggests dimensions,parameters and hypothesis.

Reality of ComplexityUntil recently, research on complex domains was limited to explorations of operationalsettings with little rigorous hypothesis testing. Now Aptima’s DDD 4.0, enables us toleverage the power of several disciplines to analyze emergent information processingsystems that are invisible from any single perspective. We simulate and explore complexdomains, precisely measure variables that will guide modeling of human informationprocessing during complex activities, and rigorously test well founded hypotheses aboutcomplex processes.

Reality ofComplexityBridge between Micro-scaleBridge between Micro-scale

Laboratory and Field ResearchLaboratory and Field Research

Bridge between Micro-scaleBridge between Micro-scaleLaboratory and Field ResearchLaboratory and Field Research

Leverage the power of several disciplines toanalyze emergent information processingsystems that are invisible from any singleperspective.

Simulate and explore complex domains.

Precisely measure variables that will guidemodeling of human information processingduring complex activities.

Rigorously test well founded hypothesesabout complex processes.

Potential Advantages overPotential Advantages overOn-the-Job-TrainingOn-the-Job-Training

Training systems in STE would be highlyTraining systems in STE would be highlyaccessible to promote frequent practice ofaccessible to promote frequent practice oflow frequency but critical events.low frequency but critical events.

Practice scenarios in STE could bePractice scenarios in STE could besystematically structured and focused onsystematically structured and focused ontraining objectives, to promote deliberatetraining objectives, to promote deliberatepractice.practice.

Performance assessments in STE could bePerformance assessments in STE could berelevant to training objectives, andrelevant to training objectives, andsystematically deliveredsystematically delivered to train andto train andmaintain competencies.maintain competencies.

Stepping Beyond EmpiricalStepping Beyond EmpiricalFoundationsFoundations

A large body of literature from cognitive and instructionalA large body of literature from cognitive and instructionalpsychology supports this approach and provides valuablepsychology supports this approach and provides valuabletraining principles (e.g. training principles (e.g. PiorolliPiorolli and Anderson, 1985; and Anderson, 1985;Schmidt and Schmidt and BjorkBjork, 1982)., 1982).

Training on tasks that are well-defined (e.g. mathematics)Training on tasks that are well-defined (e.g. mathematics)and that are executed mainly by individuals.and that are executed mainly by individuals.

In contrast, DTC functions and other command and controlIn contrast, DTC functions and other command and controloperations involve:operations involve:•• human performance by teamshuman performance by teams•• domains that are ill-defined.domains that are ill-defined.•• The DTC training under consideration is not initial training ofThe DTC training under consideration is not initial training of

new operators, but advanced training of experienced operatorsnew operators, but advanced training of experienced operatorslearning new experiences.learning new experiences.

Can such advanced training be accomplished and rigorouslyCan such advanced training be accomplished and rigorouslyinvestigated in an STE?investigated in an STE?

METHODSMETHODS ParticipantsParticipants

•• 7 undergraduate college research7 undergraduate college researchassistants (2 women and 5 men, meanassistants (2 women and 5 men, meanage =age = 22.6 years)22.6 years)

•• $7.25 per hour for 108 hours each$7.25 per hour for 108 hours each Tradeoff many participants who run few hours for fewTradeoff many participants who run few hours for few

participants who run many hoursparticipants who run many hours

•• Being research assistants motivated theBeing research assistants motivated theparticipants, but they did not know theparticipants, but they did not know thepurpose of the experiment until after itpurpose of the experiment until after itwas completed.was completed.

Audio Recording SchematicAudio Recording Schematic

Team1

Team 2

Audio Station

Hello Team

Audio Station

Four Member TeamEight Member Team

Simulated RolesSimulated Roles Five Trainees in ExperimentFive Trainees in Experiment

•• Intelligence, Surveillance, and Reconnaissance (ISR)Intelligence, Surveillance, and Reconnaissance (ISR)•• DTC: Input from ISR Output PlansDTC: Input from ISR Output Plans

DTC ChiefDTC Chief Ground Track Coordinator (GTC)Ground Track Coordinator (GTC) Attack Coordinator (AC)Attack Coordinator (AC) Target Duty Officer (TDO)Target Duty Officer (TDO)

Two Confederates in ExperimentTwo Confederates in Experiment•• Chief of Combat Operations (CCO)Chief of Combat Operations (CCO)•• Senior Offensive Duty Officer (SODO)Senior Offensive Duty Officer (SODO)

Critical roles in the larger offensive team with which theCritical roles in the larger offensive team with which theDTC interactsDTC interacts

•• Approve plansApprove plans•• Execute plansExecute plans•• Variability related to approvals and to executions wasVariability related to approvals and to executions was

minimalminimal

Represent Dynamics in OperationalRepresent Dynamics in OperationalSettingSetting

Assess

Engage

Target Track

Fix

Find

FIND

• TO1 Start

– Differentiate TST

• M0P1

– Accuracy

– Timing

TST : Time

Sensitive Target

Object of interest

identified

Object of interest tested against TST

criteria (Target Nomination list)

ISR

Nominate object of interest as

a TST

FIX (TO1: Differentiate TST)FIX (TO1: Differentiate TST)(MOP1: Accurate, Timely)(MOP1: Accurate, Timely)

TSTinformationverified

Coordinate TSTinformation

ISR

GTC

TST Validatedassigned andinformationcommunicated

CHIEF

Assess TSTagainstdynamictargetingcriteria (DTMATRIX)

AC TDO

TARGET (TO2: Plan TST priority)TARGET (TO2: Plan TST priority)(MOP2: Score given op ex of plan)(MOP2: Score given op ex of plan)

Updatedinformation ofTST provided

Evaluateweaponeeringoptions

TDOAC

GTC

Check TSTagainst NOSTRIKE List

Weapon/Targetpairing evaluated

Plancomplete DTC Chief

(TO3: Plan attack coordination)(TO3: Plan attack coordination)(MOP3: Score given op ex of plan)(MOP3: Score given op ex of plan)

Updatedinformation ofTST provided

Evaluateweaponeeringoptions

TDOAC

GTC

Check TSTagainst NOSTRIKE List

Weapon/Targetpairing evaluated

Plancomplete DTC Chief

Represent Dynamics in OperationalRepresent Dynamics in OperationalSettingSetting

Communicate by text messaging andCommunicate by text messaging andoccasionally communicate by voiceoccasionally communicate by voice

Air Force brevity procedures for bothAir Force brevity procedures for bothcommunication modescommunication modes

Uncertain information within the Uncertain information within theconstraints of an existing mission plan, theconstraints of an existing mission plan, theAir Tasking Order (ATO)Air Tasking Order (ATO)•• ATO changes every 8-12 in field and every 1 hrATO changes every 8-12 in field and every 1 hr

mission in experimentmission in experiment•• ATO includes guidance on:ATO includes guidance on:

what kinds of what kinds of TSTsTSTs might be encountered might be encountered how how TSTsTSTs should be prioritized relative to the ATO should be prioritized relative to the ATO How How TSTsTSTs should be prioritized relative to each other. should be prioritized relative to each other.

DesignDesign IV: Manipulated Within-participantIV: Manipulated Within-participant

•• pretests versus posttestspretests versus posttests•• Phase II versus Phase IIIPhase II versus Phase III

DV: Experts rate quality of the proposed strikeDV: Experts rate quality of the proposed strikepackage (TO3) for each TSTpackage (TO3) for each TST•• incorrect with high confidence = 1incorrect with high confidence = 1•• incorrect with low confidence = 2incorrect with low confidence = 2•• correct with low confidence = 3correct with low confidence = 3•• correct with high confidence = 4correct with high confidence = 4•• submitting no plan before the TST disappeared = 0submitting no plan before the TST disappeared = 0•• submitting a plan for prosecuting a target that was on submitting a plan for prosecuting a target that was on

the no strike list = -1.the no strike list = -1. The expert ratings were the same for 96% of theThe expert ratings were the same for 96% of the

ratingsratings Conferences resolved disagreementsConferences resolved disagreements

ProcedureProcedure

Phase I (50 hours) backgroundPhase I (50 hours) backgroundinformation using materials from Airinformation using materials from AirForce publicationsForce publications•• Characteristics of enemy weapons andCharacteristics of enemy weapons and

friendly assetsfriendly assets•• Communication brevity procedures,Communication brevity procedures,•• Communication dynamics among theCommunication dynamics among the

team members.team members.•• Assisted PracticeAssisted Practice

ProcedureProcedure Phase II (49 hours): trainees performedPhase II (49 hours): trainees performed

49 unassisted sessions49 unassisted sessions•• Planning (10 min): team commits to plansPlanning (10 min): team commits to plans

about ATO and possible about ATO and possible TSTsTSTs•• Mission execution (40 min): team judiciouslyMission execution (40 min): team judiciously

adapts plans when unexpected events occuradapts plans when unexpected events occur•• Debriefing (10 min): team extracts lessonsDebriefing (10 min): team extracts lessons

from previous plans and adaptations and thenfrom previous plans and adaptations and thenincorporates these lessons during subsequentincorporates these lessons during subsequentplanning, missions, and debriefsplanning, missions, and debriefs

Planning and Debrief sessions with checklistsPlanning and Debrief sessions with checklistsanalogous to those used by the Air Forceanalogous to those used by the Air Force

ProcedureProcedure

Phase III (9 hr): Trainees performedPhase III (9 hr): Trainees performed9 more unassisted missions, during9 more unassisted missions, duringwhich they trained for newwhich they trained for newexperiences.experiences.•• Enemy changed strategies to exploitEnemy changed strategies to exploit

vulnerabilities in current defensevulnerabilities in current defensestrategies (destroyed refueling tankers)strategies (destroyed refueling tankers)

•• Enemy increase their weapons Enemy increase their weapons Added after the initial effect of the strategyAdded after the initial effect of the strategy

change was measuredchange was measured Increased difficulty of learning newIncreased difficulty of learning new

strategiesstrategies

Experiment 1: Mean accuracyExperiment 1: Mean accuracyratings for training objective 3ratings for training objective 3

(TO3) on two sets of pretests and(TO3) on two sets of pretests andposttestsposttests

0

0.5

1

1.5

2

2.5

3

3.5

4

Pre II Post II Pre III Post III

TO3 ACC

Experiment 2: Mean accuracyExperiment 2: Mean accuracyratings for training objective 3ratings for training objective 3

(TO3) on two sets of pretests and(TO3) on two sets of pretests andposttestsposttests

0

0.5

1

1.5

2

2.5

3

3.5

Pre II Post II Pre III Post III

TO3 Accuracy

New Paradigm andNew Paradigm andFuture DirectionsFuture Directions

Understand effects andUnderstand effects andinteractions among theinteractions among themany aspects of combatmany aspects of combatidentificationidentification•• What an object ofWhat an object of

interest isinterest is•• Whether it is expectedWhether it is expected•• Whether it is a threat orWhether it is a threat or

opportunityopportunity•• Whether it demands aWhether it demands a

response within theresponse within thecurrent mission ascurrent mission asspecified by the ATOspecified by the ATO

•• Whether availableWhether availablefriendly assets have thefriendly assets have thecapacity to respond.capacity to respond.

New Paradigm andNew Paradigm andApplicationsApplications

Potential immediatePotential immediatebenefits to actualbenefits to actualDTC teams ofDTC teams ofsimulating a specificsimulating a specificnew experience,new experience,such as confrontingsuch as confrontinga new enemy in aa new enemy in anew contextnew context

Potential learning-Potential learning-to-learn benefits ofto-learn benefits ofsimulating multiplesimulating multiplenew enemy forces,new enemy forces,new contexts, andnew contexts, andnew enemy tactics.new enemy tactics.

New Paradigm andNew Paradigm andApplicationsApplications

In summary, theIn summary, thegoal of traininggoal of trainingactual DTC teams toactual DTC teams tolearn newlearn newexperiences in STEexperiences in STEtraining systemstraining systemsusing the newusing the newparadigm would beparadigm would beto make theirto make theircombatcombatidentification moreidentification moreadaptive to newadaptive to newexperiences inexperiences inoperationaloperationalsettings.settings.