Lessons learned from planning and preparing a distributed

UNCLASSIFIED

UNCLASSIFIED

Lessons learned from planning and preparing a distributed ISR LVC Environment

- and there were lots

John W. DiemTest Technology Director, US Army Operational Test CommandITEA Systems of Systems Engineering WorkshopJanuary 29, 2015

UNCLASSIFIED

UNCLASSIFIED

Why did we need an ISR LVC Solution for OT – and why distributed?

• For the Distributed Common Ground Station – Army (DCGS-A) Limited Users Test to be conducted as part of the Network Integration Evaluation (NIE) 15.2 in May 2015• Preceded by lab based risk reduction @ Aberdeen Proving

Grounds, MD• Preceded by developmental testing @ Fort Huachuca, AZ• Preceded by multiple Technical Integration Events for each of

the 3 preceding NIEs (14.1, 14.2, 15.1)

• Required hybrid LVC solutions from testing, training, experimentation, and operations – even 3 letter agency support – to create the right mix of simulation, scripting and control for a free-play environment for the system, its system of systems – and most importantly, its users

• Long ramp-up: multiple DTs, LBRRs, and NIEs as preparatory events – the economy and power of distributed planning, engineering, and support became readily apparent. Getting there was another matter.

Presenter

Presentation Notes

The purpose of this webinar is to show the utility of Live Virtual, Constructive (LVC) network models to the broad DoD community to include Development, Test, Plan, and Training of Networks and Network-enabled systems. We will: Discuss the importance and challenges of developing, testing and deploying tactical communications networks Review the recent advances in LVC network models and how they address these challenges Provide case studies of LVC network model use in operational test, analysis, experimentation and cyber threat assessment, and Share some thoughts on the future of LVC network modeling and simulation.

UNCLASSIFIED

UNCLASSIFIED

What does distributed mean for this test – and can it help?(according to dictionary.com)

distribute[dih-strib-yoot]

verb (used with object), distributed, distributing.1.to divide and give out in shares; deal out; allot.2.to disperse through a space or over an area; spread; scatter.3.to promote, sell, and ship or deliver (an item or line of merchandise) to individual customers, especially in a specified region or area.4.to pass out or deliver (mail, newspapers, etc.) to intended recipients.5,6.to divide into distinct phases or classes:The process was distributed into three stages.These plants are distributed into 22 classes.

Build and prove-out hybrid teams and architectures

Geography – but more: security, architectures, protocols,

nationality, standards

YES! – starting with planning and ending in test execution

Link evaluators, testers, tool developers – concept to

accreditation

Leverage SoS elements for time needed

Grow the capability from concept through LBRR through

DT through LUT to IOTE

UNCLASSIFIED

UNCLASSIFIED

How has this test been distributed?• Geography – but not always nation or world-wide. Sometimes the most

challenging link was between simulations at Fort X and live players at Range Y, 60 miles and multiple tactical and installation backbones away

• Security – unclassified live tactical engagement simulations tied to collateral constructive simulations tied to SCI simulations and operational ISR systems, working at collateral and SCI

• Architectures, Standards and Protocols – DIS, HLA, TENA, multiple tactical message formats and data exchanges distributed across an LVC federation linked at multiple points to a C4ISR enterprise as well as data collection and test control sysetms

• Responsibilities – PM, DT, OT, Evaluator for concept, multiple agencies for scenario and event control development (data, road to war, event lists,….). A Team of Teams

• Time – Multiple events over a 2 year window• System of Systems – mix of live and simulated sensors and other C4ISR

inputs working in the context of a common scenario and envionment

Presenter

Presentation Notes

The purpose of this webinar is to show the utility of Live Virtual, Constructive (LVC) network models to the broad DoD community to include Development, Test, Plan, and Training of Networks and Network-enabled systems. We will: Discuss the importance and challenges of developing, testing and deploying tactical communications networks Review the recent advances in LVC network models and how they address these challenges Provide case studies of LVC network model use in operational test, analysis, experimentation and cyber threat assessment, and Share some thoughts on the future of LVC network modeling and simulation.

UNCLASSIFIED

UNCLASSIFIED

Test Environment Design: A Team SportPlanning and Preparation

- Operational Thread Requirements - Friendly Force Structure- Mission Profile - Concept of Operations

Scenario Work Group

Intel Architecture / Threat Work

Group

Validation Exercise (VALEX)

Work Group

Technical Integration

Events

Validated Test Scenario

Mission Events List

Simulation to Mission Command Network Validated

LUT

- Mission Profile - Conventional / Insurgency- Intel Message Requirements Threat Structure- Threat Profile - HVI Story Lines

- Mission Events List

- Five Events- Simulation Interoperability- Simulation to Mission Command System Integration- Network Performance Assessment

- Complete build of Test Unit and NIE Tactical Network- Intelligence WfF technical threads (live and sim)- Fires and Maneuver WfF technical threads (live and sim)

UNCLASSIFIED

UNCLASSIFIED

Test Environment Control: Another Team Sport

• Daily Intel / Threat Sync Meeting Coordinator: BMC G2 OPFOR Commander

o Live OPFOR CDRo Constructive OPFOR CDRo Threat EW Lead

LVC Team

Execution / Assessment

• ATEC Sync Meeting Coordinator: OTC AEC Test Team

• Test Team Team Hotwash

• Daily Operations Sync Meeting Coordinator: BMC Deputy Commander OTC G3, Test Team, BMC staff, OPFOR Commander, Threat EW, LVC team

• Tasks / Outputs Refine the Master Event List Refine/Adjust individual storylines Refine/Adjust threat scheme of maneuver Synchronize Live / Constructive OPFOR

Threat Entity Live / Sim Intel Signatures Intel Source

Threat Cell #1

HVI #1 Live SIGINT - Live OPFOR communications network

GMTI - Truck (Sim)

HUMINT -TiMS Inject Live Role Player

UNCLASSIFIED

UNCLASSIFIED

To ensure the test environment is robust…1. Develop overall scenario including DCGS-A LUT Test Team

requirements2. Develop storylines3. Develop individual messages to support storylines4. Live and Simulated portions must be synchronized5. Supports BDE Operations and DIV ACE (-) (ISR)

Distributed Scenario Development

ATTICA

4th ID attacks 2/1 ADVBIED

2/1 LOG PAC attacked

Example StorylinesTACREP - OPFOR Comms

HUMINT reporting

Unit Reporting

Messages of Quality Targeting

Decision Points

BCT Operations

Plan, Prepare, Execute, AssessLessons Learned: 1) Robust is a small word but is open for lots of interpretation; 2) Synchronization is a bigger word but getting that right – and defining robust - has consumed 3 NIEs and we still have a lot of work left between multiple sites to be ready for OT

UNCLASSIFIED

UNCLASSIFIED

End State: Unit feels like they are conducting live BDE operations and are supporting the commander’s decision making process• Operational Mode Summary / Mission Profile: describes the concept of

employment, missions, and environmental conditions DCGS-A capability will encounter during the full range of military operations• Combination of Wide Area Security and Combined Arms Maneuver• Messages – describes what messages should stimulate the BDE S2• TOC/TAC jumps are executed throughout the exercise in response to

the Situation• Filling data buckets – ensure our messages properly stimulate the use of

DCGS-A; quality ensures that it is a true operational environment that is relevant to the fight

• Master Event List with dynamic scripting to adjust; REDFOR Commander (Live and Sim) can make decisions (traceability i.e. truth table); reviewed/updated daily

Getting the Scenario Right (1/2)

Lessons Learned: 1) Art & Science; 2) Insuring that the means to deliver information “filled the data buckets” correctly would have benefited from a lot more distributed engineering and integration; 3) Event control distributed across simulations, scripting and control cell could also have benefited from a lot more distributed rehearsal.

Presenter

Presentation Notes

Enable DCGS-A fuses diverse data streams to provide a commander better SA to conduct the mission. an human-driven OPFOR as part of the OT

UNCLASSIFIED

UNCLASSIFIED

Message Type (24 hour period)SIPRNet JWICS “SCI”

min max min max min max

Maneuver/OPS Mission Command

Combat Reports

All Source

Collaboration

Weather

Intel Products

Collection Management

Biometrics

GEOINT

Engineer Products

Imagery Products

FMV

Moving Target Indicator

SIGINT Operations

Reporting

HUMINT/CI Operations

Reporting

MASINT MASINT

Getting the Scenario Right (2/2)

Structured from the OMS/MP Message Requirements

Lessons Learned: 1) More trial and effort than expected on finding – and documenting “operational realism” – what information, in what sources, over what networks? 2) Close collaboration with TCM and PM required – and has been improved via distributed integration efforts pre-test.

UNCLASSIFIED

UNCLASSIFIED

Distributed LVC EnvironmentLVC Simulation Tool Functional Representation Outputs to Mission Command

One Semi-Automated Forces (OneSAF) - Friendly/Threat Ground Maneuver Systems- Threat Rocket, Artillery, and Mortar Systems- Neutral Ground Systems/Personnel

- Combat Messaging- Joint Variable Message Format (JVMF) and Data Distribution Services (DDS) combat messaging

Intelligence Model And Simulation for Evaluation (IMASE) – Simulation and Scoring Subsystem (ISSS)

- Airborne COMINT and ELINT platforms- Unmanned Airborne ISR above BCT

-TS/SCI SIGINT messaging- GEOINT interpretation messages (RECON Exploitation Report)

Multiple Unified Simulation Environment (MUSE)

- JSTARS- Unmanned Airborne ISR above BCT

-Ground Moving Target Indicator data- Full Motion Video

Joint Cryptologic Mission Simulation (JCMS) - Pulls SIGINT messaging from simulation and SIGINT scripting cell into a Real Time Regional Gateway (RTRG)

- TS/SCI SIGINT messaging

Training Brain Operations Center (TBOC) Traffic Integration Messaging System (TiMS)

- Scripted HUMINT and Open Source products - HUMINT, Open Source news feeds, other All-Source products

Extensible C4I Instrumentation Suite – Fire Support Application (ExCIS-FSA)

- Friendly Indirect Fire Platforms, Sensors, and Mission Command Nodes

- Tactical Messaging between live and simulated Field Artillery units’ AFATDS and simulated artillery firing platforms

CRAM Distributed System of Systems Simulation (CDS3)

-Friendly Air Missile Defense sensors, C2 nodes, and firing platforms- Fixed Wing Aircraft and Ballistic Missiles

- Air Defense Engagement messaging- Air Picture to BCT Air Defense Airspace Management (ADAM) Cell

The ATEC Player Evaluation Tracking System(TAPETS) integrated with MILES

- Live Player Instrumentation- Tactical Engagement, Position Location

Common Data Link (CDL) - Integrate Live Player position data from TAPETS into the constructive simulation- Live and Constructive synchronization

Lessons Learned: 1) Getting all the tools to work together – most of which we don’t own – would have benefitted from more distributed testing, up front. 2) As we got more comfortable with how the tools worked – alone and together – we began to employ more and more from their home station vice bring to the test site.

UNCLASSIFIED

UNCLASSIFIED

Distributed Live EnablersLinked to LVC and/or Tactical Systems/Networks

Enabler Functional Representation Outputs to Mission Command System

OPFOR and civilian communications network - Threat and civilian role player communications - Signature for live SIGINT assets

SIGINT White Cell (BMC G2 and Strategic Plans and Exercise Branches)

- Discrete SIGINT reporting to augment simulation feed

- COMINT messaging

ISR Integration Cell (formerly ISR TOPOFF) - COCOM ISR Collection Manager- Provides GEOINT and Imagery products- Responds to Unit RFIs

- GEOINT- COCOM ISR Collection Plan to Division/JTF HQ

Live Role Players - Represent HVI and civilian persons of interest- Interact with live forces and company response cells during live and simulated engagements

- HUMINT Sources- SIGINT Signature

Live Sensors - Organic BCT ISR - Live SIGINT and Full Motion Video

Live Maneuver Battalion - All Warfighting Functions- HUMINT Collection Teams (HCT)

- Combat Reporting - HUMINT- Tactical Questioning

Company Response Cells` - Live Company Headquarters- Tactical Communications and Mission Command Systems- Constructive maneuver platoons

- Combat Reporting- HUMINT- Tactical Questioning

Observer Controller Network - Indirect Fires adjudication- Facilitate Live OPFOR / Constructive OPFOR synchronization

Lessons Learned: 1) Some of the hardest distribution challenges are within the 60 mile radius of the test site: lack of cross-domain solutions, “stitching” multiple test support networks together, synching live sensor feeds on live force w/ simulated sensor feeds on live + constructive force; 2) Lack of a cross-domain solution (CDS) that can link LVC and operational systems on open SIPR (and higher) is moving from nuisance to mission critical; 3) Geographic boundaries only go so far when synchronizing live and simulated forces.

UNCLASSIFIED

UNCLASSIFIED

How it all comes together: Distributed ISR OT – more than geography

DistributedSims and Operational Networks for Product Distribution Lessons Learned: 1) In almost every case, the impact of disruptions to network

connectivity – the “glue of the event” - were no worse than if the LVC assets were collocated with the test. Said another way – they weren’t an excuse not to go distributed 2) Geographic boundaries only go so far when synchronizing live and simulated forces (worth saying twice).

12

UNCLASSIFIED

UNCLASSIFIED

Summary: We found real or potential benefits – and learned lessons – across the whole OT timeline

• More insightful analysis of alternatives for test plans, scenarios, architectures – “Try before you buy”

• Achieved more realistic scenarios and test environments –and may have only seen the tip of the iceberg

• Earlier identification of potential issues – and solutions -during scenario, LVC architecture, data collection, and test control development

• We were able to help PM relatively late in the cycle – could have helped avoid even more unexpected costs and delays

• Lowered the cost to prepare for and execute the OT – using distributed architecture will save roughly $400K for one event

Presenter

Presentation Notes

The result- benefits across the entire acquisition lifecycle Operational Test Community : Significant cost savings -- reduced need for live test articles & operators Reduce test costs: debug test scenarios with LVC models prior to field events Provided a robust test environment that would be cost-prohibitive to reproduce with all live assets Realistic network loading by accounting accurately for network & user traffic Common environments for test of applications, radios, or network tools Analysis Early insight into performance gaps and consistent evaluation of alternative strategies to mitigate the gaps Early insight into interoperability challenges for heterogeneous networks Consistent comparison of alternatives in operationally accurate scenarios Development Test & Program Management cost avoidance by facilitating testing in early development stages Cost avoidance by testing operational readiness and specifically interoperability gaps in DT events Cost reductions by using a shared repository of scenarios developed throughout the product lifecycle Cyber Evaluate impact of diverse cyber threats on application resilience (and/or mission assurance) from an operational perspective Evaluate impact of mitigation strategies for ‘critical’ cyber threats on application resilience A common environment for cyber threat analysis at all layers form physical, network, to application vulnerabilities Experimentation: Conduct experimentation for conditions not achievable at ranges due to safety, security, limited test assets or space available A robust environment in which to facilitate experimentation with wireless assets which is typically cumbersome in live ranges Training Add realistic communication and cyber effects by embedding network models in current generation of trainers. Facilitate kinetic & cyber training within a single framework

UNCLASSIFIED

UNCLASSIFIED

Questions

UNCLASSIFIED

UNCLASSIFIED

Networked Systems “Plug In” to a Tactically Realistic Virtual Network

Network Emulations

Data Collection & Analysis Tools

Hardware /System in the Loop Interface (HWIL/SITL)

Platforms

C5ISR

Sensors

AirspaceIntegration

DistributedSimulations

SAFs (OneSAF)TENA/HLA/

DIS InterfacesVirtual

Simulations

Tactical NetworkModel Library

Terrain / Weather

RF Propagation

Cyber ModelLibrary

Information Systems

Operates On Top of Wide Area Networks

Distributed testing capabilities and methods benefit OT during

planning, engineering, preparation, and execution.

OT Test Environment Template for Networked Systems

Presenter

Presentation Notes

At the core is the ability to emulate a real tactical network. There are multiple tools already developed that do this and Always On-On Demand will leverage the best of breed. The ability to take a tactical system, be it a radio, sensor, aircraft, C2, or anything else with a network plus … and plug it into a simulation that correctly replicates a tactical network environment opens up a world of possibilities. No longer do we have to rely on field testing to find out how a system, or combination of systems, will react to adverse conditions where weather, terrain, spectrum, etc, will reduce or stop network traffic from one location to another. The core (at the center) is the model of the tactical network; the environmental factors that affect wireless signal propagation including terrain, mobility, interference .. are also modeled accurately. Statistics and relevant data on the mission is collected and analyzed using the identical (and familiar) set of tools used with live networks. Standard interfaces are used to link the LVC network model with simulators of other system components.

Documents

Lessons learned from planning and preparing a distributed