Upload
others
View
36
Download
3
Embed Size (px)
Citation preview
Test amp Evaluation
Challenges for the Future
Chip Ferguson
Joint Mission Environment Test Capability
Program Manager May 11 2011
2
DoD Strategic Plan
for TampE Resources
Service
TampE Needs and
Solutions Process
Service
TampESampT
Working Groups
Relationship within TRMC
Synergy through Aligned Investment
Quadrennial Defense Review
Strategic Planning Guidance
Service Improvement
amp Modernization
Programs
Acquisition Programs
Advanced Concept
Technology Demonstrations
TampE Multi-Service
Agency
Capabilities
Risk mitigation needs Technology shortfalls
Risk mitigation solutions
Advanced development
Requirements
Capabilities
DoD Corporate
Distributed Test
Capability
TRMC Joint Investment Programs
63 64 65
Distributed Testing
A process preferably persistent and continuous for linking
various geographically separated Live Virtual and
Constructive sites and systems together in a distributed
environment for use across the acquisition life cycle to
support and conduct the Test and Evaluation (TampE) of a
system or systems-of-systems
GOAL Near Real-Time Test-Fix-Test
across organizational boundaries
3
Why Consider Distributed Test
bull Is there a requirement to exchange data within your system or within a
system-of-systems (SoS)
bull Do you have a requirement to address SoS interoperability issues early
in the acquisition process
bull Do you need more systems under test for live testing than what is
available at your facility
bull Do you have all of the appropriate ldquosupporting castrdquo (supporting
systems C4ISR assets etc) needed for testing
bull Do you need additional threat types fidelity and density in realistic
numbers at realistic ranges for testing
bull Do you need to incorporate TampE into training events
4
If you are answering lsquoNOrsquo to any of these
questions you will soon be saying lsquoYESrsquo
When Is Distributed Test Appropriate
bull Examples Where Appropriate
ndash Interoperability testing
bull C4 Interoperability with higher lower and adjacent Joint force organizations
ndash Data exchange in early DT testing
bull Interaction between sub-systems (latency may be a consideration)
bull Interaction between systems in a realistic environment
o Provide the most realistic environment possible from concept exploration through
Follow-On TampE
ndash When it is too costly to bring all the player systems together on a single
range
ndash Gain or increase operational relevance
bull Virtual and Constructive capabilities to supplement live systems (eg red
forces white forces terrain immobile test assets)
bull Examples Where Inappropriate
ndash System performance tests that do not include other systemssubsystems
ndash Reliability testing
5
Reduces Cost Risk and Time
6
JMETC Enabled Distributed Testing
Systems
Under
Test
Joint Operational Scenarios
Virtual
Prototype
Range Environment
Generator
Threat
Systems
Integrated Test
Resources
TENA Standard Interface
Definitions
TENA
Common
Middleware
TENA Standard Interface
Definitions
TENA
Common
Middleware
TENA Standard Interface
Definitions
TENA
Common
Middleware
TENA Standard Interface
Definitions
TENA
Common
Middleware
Reuse Repository
Distributed Test Support Tools
JMETC
Infrastructure
Hardware in the Loop Lab
TENA
Common
Middleware
Installed Systems
Test Facility
Basic Interface
Standard
TENA
Common
Middleware
Data Management Solutions
TENA
Standard Interface
Definitions
TENA
Standard Interface
Definitions
JMETC
Network on
SDREN
6
Distributed TampE Supports
Systems Engineering (SE)
Hardware Fabrication
ComponentInterface Definition
Allocated Functions amp Performance Requirements
Requirements Definitions (JCIDS) Transition
Requirements Analysis
Logical Analysis
Design Solution
Integration
Verification
Validation
Implementation
Tech Data amp Training Pubs
Software CreationCoding
Detailed Design
Confirmation Audits
Realization amp Assessment
Decomposition amp Definition
Input Output to Next Phase of
Development or Lifecycle Support Operational Need
Measures of EffectivenessSuitability
System Measures of Performance
Element Design Criteria
Spec
ialt
y En
gin
eeri
ng
Dev
elo
pm
ent
Des
ign
En
gin
eeri
ng
Dec
isio
n
Au
tho
rity
Tech Readiness
Assessment
Early TampE
Involvement
Component Level
Testing
Component
Characterization
Early Operational
Assessment
DTampE
Verification
OTampE
Validation
Source DAU Acquisition Community Connection
Early Injection of
Authoritative
systems and
mission threads
Combined DTampE
OTampE in a Joint
or SoS
Environment
Persistently Integrated
but disparately
developed capabilities
8
Sustainment Systems Acquisition (Engineering amp manufacturing development
demonstration LRIP amp production)
Pre-Systems Acquisition
IOC A B
Concept amp Tech Development
Concept
Exploration
Component
Advanced
Development
Decision
Review
System
Integration
Interim Progress Review
Production amp Deployment
LRIP
FRP Decision Review
Full-Rate Production amp Deployment
Operations
amp Support
C
The Future of Distributed Testing Test-Fix-Test
Across the Acquisition Life Cycle
Rapid Acquisition Developmental Test Operational
Test Interoperability Certification Net-Ready Key
Performance Parameters testing Joint Mission
Capability Portfolio testing
JMETC enables continuous testing across the acquisition life cycle JMETC reduces acquisition time and cost By Providing
bull Readily-available
persistent connectivity
bull Standing network
security agreements
bull Common
interoperability software
for linking sites
bull Certified test tools for
distributed testing
Enables early verification that
systems work stand alone and in a
Joint Environment
Helps find problems early in
acquisition ndash when they are less
costly to fix
Outline Distributed
Testing and JMETC
requirements in TEMP
Creates robust environment for common prototype analysis
Provides subject matter expertise to integrate distributed facilities
Test Early in Acquisition Process
Recent Distributed Test at a major test facility
bull C2 Interoperability upgrade to National Strategic asset
bull Significant software and interoperability problems found during initial Distributed Testing
bull Discovered system NOT ready for open air testing IMPACTmdashRisk Reduction Early
Identification of Problem
bullEarly Distributed Testing led to early
identification and correction of
interoperability issues
bullNo range or flying costs 9
Post-Milestone A ndash Concept Development
Joint Surface Warfare JCTD
Description
Point Mugu Test Team
demonstrated Net Enabled
Weapon Link-16 capability
using FA-18EF as launch
platform JSOW C-1 as
weapon and JSTARS as 3rd
party target source
Sample of Measures
Tactical message timing
between the controller
and the weapon
IMPACT-- Efficiency Lower Technical
Risk and Cost Savings
bull Program scheduled and executed short
multiple tests for incremental software
update evaluation
bull Resources expended on test amp analysis and
not network setup and monitoring
China Lake FA-18 JSOW
10
11
Description bull Assessment of B-2 Link-16
interoperability with AWACS
bull Connected live B-2 on ramp
at Whiteman AFB MO an
AWACS HITL at Tinker AFB
OK within a distributed C2
environment
IMPACT-- Cost Savings and Better
Product
bull Early testing led to early identification
and correction of Link 16 interoperability
issues
bull No range or flying costs
Army
Air Force
Navy
Marines
Joint
Industry
As of 14 Sep 2009
Tinker AFB AWACS
Whiteman B-2
Eglin 46TS
WPAFB SIMAF
JMETC
Langley GCIC
Army
Air Force
Navy
Marines
Joint
Industry
As of 14 Sep 2009
Tinker AFB AWACS
Whiteman B-2
Eglin 46TS
WPAFB SIMAF
JMETC
Langley GCIC
Post-Milestone C ndash Developmental Test
B-2 Spirit ICE Data Link Test
Sample of Measures
Timeliness and accuracy
of message exchanges
bull Joint Urgent Operational Need
bull Integration of BACN payload
onto multiple platforms for
solution to net-centric warfare
challenges of
bull Beyond line-of-sight comm
bull Relay bridge and range
extension for ground forces
and supporting aircraft
bull Distributed Testing included
Live-fly DT and Operational
Utility Evaluation Fall of 2010
IMPACT -- Cost Savings and
Better Product bullEfficient integration of DT and OT
bullAll required assets not available on-site
bullDistributed Testing saved ldquo$12Mrdquo
bullUrgent capability fielded
12
Responsive TampE for Rapid Fielding
Battlefield Airborne Communication Node
(BACN)
Challenges We Are Overcoming Today
Challenge
bull Site Interconnectivity
bull No common enabling
transport architecture
bull Inconsistent Tools
bull Test Participant
Collaboration
bull Analysis of Disparately
Stored Data
Overcome By
JMETC Network
TENA Middleware
JMETC ldquoBest of Breedrdquo
Process
Reuse Repository
Data Management Solutions
JMETC Joint Mission Environment Test Capability
TENA Test and Training Enabling Architecture
A TRMC Perspective
We Need a Common Vision for
Test and Training Infrastructure
Mr Derrick Hinton Principal Deputy Director Test Resource Management Center (TRMC)
Deputy Director Joint Investment Programs amp Policy (JIPP)
Office of the Undersecretary of Defense for Acquisition Technology amp Logistics (OUSD[ATampL])
Annual ITEA Test Instrumentation Workshop XIV
Test and Training Enterprise
12 May 2010
2010 2030
Individual Systems WHAT do we test Mission-Level
amp Systems of Systems Capabilities
Events WHEN do we test Continuously
Decision-Makers For WHOM do we test Decision-Makers
amp Warfighters
Agile
Streamlined
Affordable
Continuous
Goal Distributed Integrated Testing and Training
The DoD TampE process must evolve to be
Test Resource Management Center
Future TampE Process
15
Aspects of the 2030 Common
Vision
bull Test amp Training bull Real-Time TTP SOP Development to improve SoS effectiveness and
suitability (limited)
bull ldquoBlack boxrdquo and ldquowhite boxrdquo testing done in parallel
including
bull System functionality
bull System effectiveness
bull System of Systems effectiveness via mission threads in an
operationally relevant environment
bull Fewer test runsconfigurations with more meaningful
collected data
bull Real-Time Data Analysis amp Automated Reporting
OBJECTIVE Near Real-Time Test-Fix-Test 16
Some Challenges to Achieving
2030 Vision
bull Authoritative Joint Mission threads
bull Mission level measures
bull Thorough but Streamlined Information Assurance
Procedures
bull Enterprise-level Multi-Level Security solutions
bull Embedded Encryption on acquisition systems
bull RF over the wire (network)
bull Near Real Time Test-Fix-Test
bull Data Management
bull Embedded amp Unobtrusive Instrumentation
bull World wide wireless
bull Extremely efficient usage of the frequency spectrum 17
Examples Where the Vision Is Starting
to Come Together
bull Joint Integrated Air amp Missile Defense Organization
(JIAMDO) Correlation Decorrelation Interoperability Test
(CDIT) is reducing analysis cycle from 3 months to less
than a month
bull Changes to test configuration between test runs (~30 minutes)
bull Overnight AdditionSubtraction of test runs depending on issues
discovered and data analysis results
bull Real-time impact to test configuration and number of test runs
bull Emphasis on evaluators and tools for detailed data analysis
bull Air-Ground Integrated Layer Exploration (AGILE) bull AGILE provides an LVC capability for acquisition amp testing
bull Analytical Rigor provides decision quality data for decision support
bull Efficient amp collaborative use of Joint analytical resources (in
addition to service capabilities)
bull Sustains and evolves collaborative environment for future use 18
Some TRMC Investments for
Achieving Common Vision
bull Central TampE Investment Program (CTEIP)
bull Test and Training Enabling Architecture (TENA)
bull Multi-Level Security-Joint Coalition Network Environment (MLS-
JCNE)
bull InterTEC
bull Result JITC-certified instrumentation usable in all aspects of TampE
bull Common Range Integrated Instrumentation System (CRIIS)
bull TampE SampT Program
bull TENA in a Resource Constrained Environment (TRCE)
bull NST Evaluation Capability Model (NECM)
bull JMETC Program
bull DoD corporate approach for distributed testing 19
What Can You Do to Be Ready for
2030
bull Maintain Information Assurance agreements for all facility
systems
bull Streamline data collection amp analysis at your facility
bull Get comfortable with DoDAF views
bull Look across the RDTampE the community to find common
problems that can be addressed with similar solutions
bull TENA-enable your instrumentation systems and test
command amp control
bull Get involved share your requirements with your Service
representatives and in TRMC users meetings
20
JMETC Users Group
bull Purpose is to focus on technical requirements and solutions relevant to current and future Distributed Testing needs
bull Technical and Management level representatives identify core infrastructure requirements and most importantly resolve issues
bull Quarterly meetings of 250-300 JMETC customers acquisition programs test events ranges LVC sites tools and network providers
21
Last JMETC Users Group Meeting bull June 2011 bull Location Mid-East Coast bull Tracks
bull User Requirements bull Information Assurance
Security bull Data Management bull InterTEC (Current amp Planned) bull Networking
bull An established forum for the Distributed Test Community to
o Identify core infrastructure requirements and use cases
o Identify investigate amp resolve issues
o Identify opportunities to collaborate
o Discuss available solutions tools and techniques
o Share lessons learned
Summary
bull We are becoming more efficient in how we execute TampE
bull We must get the biggest and best ldquobang for the buckrdquo that we can
bull TRMC and Services continue TampE investments to enable realization of the 2030 Vision
bull Align TampE (OSD and Services) investments to mitigate test risk areas
bull The Future of TampE will include Distributed Integrated Testing and Training and will enable near real time Test-Fix-Test
22
JMETC Program Points of Contact
23
JMETC Program Manager Chip Ferguson chipfergusonosdmil 703-601-5274
JMETC Principal Deputy PM Bruce Bailey brucebaileyosdmil 703-601-5208
JMETC Lead Operations Planning Marty Arnwine martemasarnwineosdmil 703-601-5215
JMETC Senior Technical Advisor George Rumford georgerumfordosdmil
703-601-5233
JMETC Lead Systems Engineer Ryan Norman ryannormanosdmil
703-601-5277
JMETC Lead Network Engineer Arjuna ldquoAJrdquo Pathmanathan ArjunaPathmanathanosdmil
703-601-5214
JMETC Website wwwjmetcorg
Backup Slides
25
FY10 JMETC Events
Customer Event Record Event Dates
Air Force JEFX 10-1Spirit Ice (B2 Data Link Test) October ndash November 2009
Navy BAMS LVC DE October ndash December 2009
Air Force Battlefield Airborne Comm Node (BACN) JUON
(DTOT)
November 2009 to September 2010
Air Force Agile Fire 10-2 January 2010
Air Force JEFX 10-23 FebruaryApril 2010
Joint Joint Surface Warfare JCTD February to September 2010
JS J8JIAMDO Joint Sensor Integration April to September 2010
Air Force B1-B Fully Integrated Data Link Testing April 2010
JFCOM J8489 (TEST WEEK) JCAS Distributed Test June 2010
JIAMDO (Navy Lead) CorrelationDecorrelation Interoperability Test
(CDIT) Integration Events (Continuous)
July to September 2010
Army (Lead) UAS in the National Airspace July to September 2010
Air Force Agile Fire 10-3 August 2010
Joint JITC Joint Interoperability Tests JIT 10-3 amp 11-1
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Joint Strike Fighter (JSF) JIAMDOJoint Track Manager
Brigade Combat Team (BCT)
Modernization
Multi-Function Adv Data Link (MADL) Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run 25
26
JMETC FY10 Accomplishments
JMETC Accomplishments
Supported 88 distinct customer test activities
Expanded network from 38 to 57 sites
ATIN and JTDL Networks transitioned to JMETC
Upgraded JMETC support applications and utilities to TENA R6
DIACAP Tiger Team report completed and recommendations being executed
Enhanced JMETC services and capabilities provided by leveraging InterTEC Services and Industry
Reuse Repository usability improvements
FY10 Example JMETC Customers
Joint Integrated Air amp Missile Defense Organization (JIAMDO)
Broad Area Communications Node (BACN) JUON
B1-B
Broad Area Maritime Surveillance System (BAMS)
Air-Ground Integrated Layer Exploration (AGILE)
Joint Interoperability Test Command (JITC)
Selected Benefits to the DoD
Integrated DT amp OT on a Joint Urgent Operational Need for the warfighter
Maximized usage of theater assets during limited maintenance windows
Improved Joint track information sharing to ensure interoperability of systems in theater operations
Coalition exchange and examination of real-time air picture data
Identification of Air Force Initiatives ready for warfighter transition
Investigated tactical UAS deployment in the National Airspace
Employment of Net-Enabled Weapons
JCAS immediate request amp end-to-end processes ldquoas-isrdquo characterization
Determined distributed system components were not ready for full live integration testing
Executed testing to support system-of-system interoperability certification
27
Customer Event Record Event Dates
Joint JIAMDO CorrelationDecorrelation Interoperability Test (CDIT)
Integration Events (Continuous) OCONUS October 2010
Navy UAS in NAS Runs for Record October 2010
Internal InterTEC Spiral 3 Systems Acceptance Test October - November 2010
Joint JITC Joint Interoperability Tests JIT 11-12345 (Continuous) October 2010 ndash September 2011
Air Force B1-B Fully Integrated Data Link Testing November 2010
Air Force AGILE Fire Phase III JEFX 2011 December 2010 - February 2011
Joint JTRS JPO -- JTRS Ground Mobile Radio January 2011
Navy Broad Area Maritime Surveillance (BAMS) (Continuous) January ndash September 2011
Joint Joint Track Manager Capability Demonstration (Continuous) January ndash September 2011
Joint JSF Initial MampS Interoperability (Continuous) February ndash March 2011
Air Force JSTARS Interoperability Test May 2011
Joint JS J8JIAMDO Joint Sensor Integration June ndash August 2011
FY11 JMETC Events
(More to Come)
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Global Hawk GATOR
Brigade Combat Team (BCT)
Modernization F-22 FY12 Testing Planned Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run
28
JMETC Connectivity
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca (2) JITC JTDL
Redstone (7) SMDC
RTC DTCC GMAN
SED Patriot THAAD FAAD SIV
Charleston (2)IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB SIMAF
As of 16 Mar 2011
0
10
20
30
40
50
60
70
FY06 FY07 FY08 FY09 FY10 FY11
JMETC Site Expansion
Sites
Dedicated trusted connectivity on SDREN (part of the GIG)
Encrypted for Secret ndash System High
DISA-registered IP address space
Active monitoring of network performance
Capable of supporting multiple simultaneous test events Bethpage NGC BAMS
Whiteman B-2
Eglin AFB (4)AOC DTF GWEF KHILS
Hurlburt Field C2DAC
Sites in Gulf Range
Functional Sites 61
New Sites Planned 8
Connection Points to Other Networks 5
Ft Greely CRTC
Site in Alaska
MHPCC
PMRF Bldg 105
Sites in Hawaii
Camp PendletonMCTSSA
China Lake (3)AV-8B FA-18IBAR
Edwards Ridley
Corona NSWC
El Segundo NGC B-2
Sites in SoCal
Point Loma (2)RLBTSSSC-PAC 59140 SSC-PAC HAIPE
Point Mugu (2)ITEC EA-6B
West Agg Rtr
Rancho Bernardo NGC BAMS
Pax River (5)ESTEL E2CD MCLACETEF SAIL ATR JMETC SYSCONEast Agg Rtr
Aberdeen ACCN
Langley (2) C-GIIF TDLITC
JFCOM JSIC
Wallops Island (2) SCSC SSDS
DISA Sky 7
Pentagon WARCAP
Dahlgren (2) CEDLIWSL
Dam Neck CDSA
Newport News NGC VASCIC
Sites in MD DC VA
Norfolk COMOPTEVFOR
McLean MITRE NCEL
Ft Huachuca JTRS
Bethpage NGC BCN
How an Air Force Customer Sees
Distributed Connectivity
Hardware-in-the-Loop (HWIL) Availability
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca JITC
Redstone (3) DTCC GMAN SED
Charleston (2) IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB
SIMAF
Bethpage NG BAMS
Whiteman B-2
MHPCC
PMRF Bldg 105
Sites in Hawaii
All linked by JMETC
29
Challenges Explained in Detail
30
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
2
DoD Strategic Plan
for TampE Resources
Service
TampE Needs and
Solutions Process
Service
TampESampT
Working Groups
Relationship within TRMC
Synergy through Aligned Investment
Quadrennial Defense Review
Strategic Planning Guidance
Service Improvement
amp Modernization
Programs
Acquisition Programs
Advanced Concept
Technology Demonstrations
TampE Multi-Service
Agency
Capabilities
Risk mitigation needs Technology shortfalls
Risk mitigation solutions
Advanced development
Requirements
Capabilities
DoD Corporate
Distributed Test
Capability
TRMC Joint Investment Programs
63 64 65
Distributed Testing
A process preferably persistent and continuous for linking
various geographically separated Live Virtual and
Constructive sites and systems together in a distributed
environment for use across the acquisition life cycle to
support and conduct the Test and Evaluation (TampE) of a
system or systems-of-systems
GOAL Near Real-Time Test-Fix-Test
across organizational boundaries
3
Why Consider Distributed Test
bull Is there a requirement to exchange data within your system or within a
system-of-systems (SoS)
bull Do you have a requirement to address SoS interoperability issues early
in the acquisition process
bull Do you need more systems under test for live testing than what is
available at your facility
bull Do you have all of the appropriate ldquosupporting castrdquo (supporting
systems C4ISR assets etc) needed for testing
bull Do you need additional threat types fidelity and density in realistic
numbers at realistic ranges for testing
bull Do you need to incorporate TampE into training events
4
If you are answering lsquoNOrsquo to any of these
questions you will soon be saying lsquoYESrsquo
When Is Distributed Test Appropriate
bull Examples Where Appropriate
ndash Interoperability testing
bull C4 Interoperability with higher lower and adjacent Joint force organizations
ndash Data exchange in early DT testing
bull Interaction between sub-systems (latency may be a consideration)
bull Interaction between systems in a realistic environment
o Provide the most realistic environment possible from concept exploration through
Follow-On TampE
ndash When it is too costly to bring all the player systems together on a single
range
ndash Gain or increase operational relevance
bull Virtual and Constructive capabilities to supplement live systems (eg red
forces white forces terrain immobile test assets)
bull Examples Where Inappropriate
ndash System performance tests that do not include other systemssubsystems
ndash Reliability testing
5
Reduces Cost Risk and Time
6
JMETC Enabled Distributed Testing
Systems
Under
Test
Joint Operational Scenarios
Virtual
Prototype
Range Environment
Generator
Threat
Systems
Integrated Test
Resources
TENA Standard Interface
Definitions
TENA
Common
Middleware
TENA Standard Interface
Definitions
TENA
Common
Middleware
TENA Standard Interface
Definitions
TENA
Common
Middleware
TENA Standard Interface
Definitions
TENA
Common
Middleware
Reuse Repository
Distributed Test Support Tools
JMETC
Infrastructure
Hardware in the Loop Lab
TENA
Common
Middleware
Installed Systems
Test Facility
Basic Interface
Standard
TENA
Common
Middleware
Data Management Solutions
TENA
Standard Interface
Definitions
TENA
Standard Interface
Definitions
JMETC
Network on
SDREN
6
Distributed TampE Supports
Systems Engineering (SE)
Hardware Fabrication
ComponentInterface Definition
Allocated Functions amp Performance Requirements
Requirements Definitions (JCIDS) Transition
Requirements Analysis
Logical Analysis
Design Solution
Integration
Verification
Validation
Implementation
Tech Data amp Training Pubs
Software CreationCoding
Detailed Design
Confirmation Audits
Realization amp Assessment
Decomposition amp Definition
Input Output to Next Phase of
Development or Lifecycle Support Operational Need
Measures of EffectivenessSuitability
System Measures of Performance
Element Design Criteria
Spec
ialt
y En
gin
eeri
ng
Dev
elo
pm
ent
Des
ign
En
gin
eeri
ng
Dec
isio
n
Au
tho
rity
Tech Readiness
Assessment
Early TampE
Involvement
Component Level
Testing
Component
Characterization
Early Operational
Assessment
DTampE
Verification
OTampE
Validation
Source DAU Acquisition Community Connection
Early Injection of
Authoritative
systems and
mission threads
Combined DTampE
OTampE in a Joint
or SoS
Environment
Persistently Integrated
but disparately
developed capabilities
8
Sustainment Systems Acquisition (Engineering amp manufacturing development
demonstration LRIP amp production)
Pre-Systems Acquisition
IOC A B
Concept amp Tech Development
Concept
Exploration
Component
Advanced
Development
Decision
Review
System
Integration
Interim Progress Review
Production amp Deployment
LRIP
FRP Decision Review
Full-Rate Production amp Deployment
Operations
amp Support
C
The Future of Distributed Testing Test-Fix-Test
Across the Acquisition Life Cycle
Rapid Acquisition Developmental Test Operational
Test Interoperability Certification Net-Ready Key
Performance Parameters testing Joint Mission
Capability Portfolio testing
JMETC enables continuous testing across the acquisition life cycle JMETC reduces acquisition time and cost By Providing
bull Readily-available
persistent connectivity
bull Standing network
security agreements
bull Common
interoperability software
for linking sites
bull Certified test tools for
distributed testing
Enables early verification that
systems work stand alone and in a
Joint Environment
Helps find problems early in
acquisition ndash when they are less
costly to fix
Outline Distributed
Testing and JMETC
requirements in TEMP
Creates robust environment for common prototype analysis
Provides subject matter expertise to integrate distributed facilities
Test Early in Acquisition Process
Recent Distributed Test at a major test facility
bull C2 Interoperability upgrade to National Strategic asset
bull Significant software and interoperability problems found during initial Distributed Testing
bull Discovered system NOT ready for open air testing IMPACTmdashRisk Reduction Early
Identification of Problem
bullEarly Distributed Testing led to early
identification and correction of
interoperability issues
bullNo range or flying costs 9
Post-Milestone A ndash Concept Development
Joint Surface Warfare JCTD
Description
Point Mugu Test Team
demonstrated Net Enabled
Weapon Link-16 capability
using FA-18EF as launch
platform JSOW C-1 as
weapon and JSTARS as 3rd
party target source
Sample of Measures
Tactical message timing
between the controller
and the weapon
IMPACT-- Efficiency Lower Technical
Risk and Cost Savings
bull Program scheduled and executed short
multiple tests for incremental software
update evaluation
bull Resources expended on test amp analysis and
not network setup and monitoring
China Lake FA-18 JSOW
10
11
Description bull Assessment of B-2 Link-16
interoperability with AWACS
bull Connected live B-2 on ramp
at Whiteman AFB MO an
AWACS HITL at Tinker AFB
OK within a distributed C2
environment
IMPACT-- Cost Savings and Better
Product
bull Early testing led to early identification
and correction of Link 16 interoperability
issues
bull No range or flying costs
Army
Air Force
Navy
Marines
Joint
Industry
As of 14 Sep 2009
Tinker AFB AWACS
Whiteman B-2
Eglin 46TS
WPAFB SIMAF
JMETC
Langley GCIC
Army
Air Force
Navy
Marines
Joint
Industry
As of 14 Sep 2009
Tinker AFB AWACS
Whiteman B-2
Eglin 46TS
WPAFB SIMAF
JMETC
Langley GCIC
Post-Milestone C ndash Developmental Test
B-2 Spirit ICE Data Link Test
Sample of Measures
Timeliness and accuracy
of message exchanges
bull Joint Urgent Operational Need
bull Integration of BACN payload
onto multiple platforms for
solution to net-centric warfare
challenges of
bull Beyond line-of-sight comm
bull Relay bridge and range
extension for ground forces
and supporting aircraft
bull Distributed Testing included
Live-fly DT and Operational
Utility Evaluation Fall of 2010
IMPACT -- Cost Savings and
Better Product bullEfficient integration of DT and OT
bullAll required assets not available on-site
bullDistributed Testing saved ldquo$12Mrdquo
bullUrgent capability fielded
12
Responsive TampE for Rapid Fielding
Battlefield Airborne Communication Node
(BACN)
Challenges We Are Overcoming Today
Challenge
bull Site Interconnectivity
bull No common enabling
transport architecture
bull Inconsistent Tools
bull Test Participant
Collaboration
bull Analysis of Disparately
Stored Data
Overcome By
JMETC Network
TENA Middleware
JMETC ldquoBest of Breedrdquo
Process
Reuse Repository
Data Management Solutions
JMETC Joint Mission Environment Test Capability
TENA Test and Training Enabling Architecture
A TRMC Perspective
We Need a Common Vision for
Test and Training Infrastructure
Mr Derrick Hinton Principal Deputy Director Test Resource Management Center (TRMC)
Deputy Director Joint Investment Programs amp Policy (JIPP)
Office of the Undersecretary of Defense for Acquisition Technology amp Logistics (OUSD[ATampL])
Annual ITEA Test Instrumentation Workshop XIV
Test and Training Enterprise
12 May 2010
2010 2030
Individual Systems WHAT do we test Mission-Level
amp Systems of Systems Capabilities
Events WHEN do we test Continuously
Decision-Makers For WHOM do we test Decision-Makers
amp Warfighters
Agile
Streamlined
Affordable
Continuous
Goal Distributed Integrated Testing and Training
The DoD TampE process must evolve to be
Test Resource Management Center
Future TampE Process
15
Aspects of the 2030 Common
Vision
bull Test amp Training bull Real-Time TTP SOP Development to improve SoS effectiveness and
suitability (limited)
bull ldquoBlack boxrdquo and ldquowhite boxrdquo testing done in parallel
including
bull System functionality
bull System effectiveness
bull System of Systems effectiveness via mission threads in an
operationally relevant environment
bull Fewer test runsconfigurations with more meaningful
collected data
bull Real-Time Data Analysis amp Automated Reporting
OBJECTIVE Near Real-Time Test-Fix-Test 16
Some Challenges to Achieving
2030 Vision
bull Authoritative Joint Mission threads
bull Mission level measures
bull Thorough but Streamlined Information Assurance
Procedures
bull Enterprise-level Multi-Level Security solutions
bull Embedded Encryption on acquisition systems
bull RF over the wire (network)
bull Near Real Time Test-Fix-Test
bull Data Management
bull Embedded amp Unobtrusive Instrumentation
bull World wide wireless
bull Extremely efficient usage of the frequency spectrum 17
Examples Where the Vision Is Starting
to Come Together
bull Joint Integrated Air amp Missile Defense Organization
(JIAMDO) Correlation Decorrelation Interoperability Test
(CDIT) is reducing analysis cycle from 3 months to less
than a month
bull Changes to test configuration between test runs (~30 minutes)
bull Overnight AdditionSubtraction of test runs depending on issues
discovered and data analysis results
bull Real-time impact to test configuration and number of test runs
bull Emphasis on evaluators and tools for detailed data analysis
bull Air-Ground Integrated Layer Exploration (AGILE) bull AGILE provides an LVC capability for acquisition amp testing
bull Analytical Rigor provides decision quality data for decision support
bull Efficient amp collaborative use of Joint analytical resources (in
addition to service capabilities)
bull Sustains and evolves collaborative environment for future use 18
Some TRMC Investments for
Achieving Common Vision
bull Central TampE Investment Program (CTEIP)
bull Test and Training Enabling Architecture (TENA)
bull Multi-Level Security-Joint Coalition Network Environment (MLS-
JCNE)
bull InterTEC
bull Result JITC-certified instrumentation usable in all aspects of TampE
bull Common Range Integrated Instrumentation System (CRIIS)
bull TampE SampT Program
bull TENA in a Resource Constrained Environment (TRCE)
bull NST Evaluation Capability Model (NECM)
bull JMETC Program
bull DoD corporate approach for distributed testing 19
What Can You Do to Be Ready for
2030
bull Maintain Information Assurance agreements for all facility
systems
bull Streamline data collection amp analysis at your facility
bull Get comfortable with DoDAF views
bull Look across the RDTampE the community to find common
problems that can be addressed with similar solutions
bull TENA-enable your instrumentation systems and test
command amp control
bull Get involved share your requirements with your Service
representatives and in TRMC users meetings
20
JMETC Users Group
bull Purpose is to focus on technical requirements and solutions relevant to current and future Distributed Testing needs
bull Technical and Management level representatives identify core infrastructure requirements and most importantly resolve issues
bull Quarterly meetings of 250-300 JMETC customers acquisition programs test events ranges LVC sites tools and network providers
21
Last JMETC Users Group Meeting bull June 2011 bull Location Mid-East Coast bull Tracks
bull User Requirements bull Information Assurance
Security bull Data Management bull InterTEC (Current amp Planned) bull Networking
bull An established forum for the Distributed Test Community to
o Identify core infrastructure requirements and use cases
o Identify investigate amp resolve issues
o Identify opportunities to collaborate
o Discuss available solutions tools and techniques
o Share lessons learned
Summary
bull We are becoming more efficient in how we execute TampE
bull We must get the biggest and best ldquobang for the buckrdquo that we can
bull TRMC and Services continue TampE investments to enable realization of the 2030 Vision
bull Align TampE (OSD and Services) investments to mitigate test risk areas
bull The Future of TampE will include Distributed Integrated Testing and Training and will enable near real time Test-Fix-Test
22
JMETC Program Points of Contact
23
JMETC Program Manager Chip Ferguson chipfergusonosdmil 703-601-5274
JMETC Principal Deputy PM Bruce Bailey brucebaileyosdmil 703-601-5208
JMETC Lead Operations Planning Marty Arnwine martemasarnwineosdmil 703-601-5215
JMETC Senior Technical Advisor George Rumford georgerumfordosdmil
703-601-5233
JMETC Lead Systems Engineer Ryan Norman ryannormanosdmil
703-601-5277
JMETC Lead Network Engineer Arjuna ldquoAJrdquo Pathmanathan ArjunaPathmanathanosdmil
703-601-5214
JMETC Website wwwjmetcorg
Backup Slides
25
FY10 JMETC Events
Customer Event Record Event Dates
Air Force JEFX 10-1Spirit Ice (B2 Data Link Test) October ndash November 2009
Navy BAMS LVC DE October ndash December 2009
Air Force Battlefield Airborne Comm Node (BACN) JUON
(DTOT)
November 2009 to September 2010
Air Force Agile Fire 10-2 January 2010
Air Force JEFX 10-23 FebruaryApril 2010
Joint Joint Surface Warfare JCTD February to September 2010
JS J8JIAMDO Joint Sensor Integration April to September 2010
Air Force B1-B Fully Integrated Data Link Testing April 2010
JFCOM J8489 (TEST WEEK) JCAS Distributed Test June 2010
JIAMDO (Navy Lead) CorrelationDecorrelation Interoperability Test
(CDIT) Integration Events (Continuous)
July to September 2010
Army (Lead) UAS in the National Airspace July to September 2010
Air Force Agile Fire 10-3 August 2010
Joint JITC Joint Interoperability Tests JIT 10-3 amp 11-1
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Joint Strike Fighter (JSF) JIAMDOJoint Track Manager
Brigade Combat Team (BCT)
Modernization
Multi-Function Adv Data Link (MADL) Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run 25
26
JMETC FY10 Accomplishments
JMETC Accomplishments
Supported 88 distinct customer test activities
Expanded network from 38 to 57 sites
ATIN and JTDL Networks transitioned to JMETC
Upgraded JMETC support applications and utilities to TENA R6
DIACAP Tiger Team report completed and recommendations being executed
Enhanced JMETC services and capabilities provided by leveraging InterTEC Services and Industry
Reuse Repository usability improvements
FY10 Example JMETC Customers
Joint Integrated Air amp Missile Defense Organization (JIAMDO)
Broad Area Communications Node (BACN) JUON
B1-B
Broad Area Maritime Surveillance System (BAMS)
Air-Ground Integrated Layer Exploration (AGILE)
Joint Interoperability Test Command (JITC)
Selected Benefits to the DoD
Integrated DT amp OT on a Joint Urgent Operational Need for the warfighter
Maximized usage of theater assets during limited maintenance windows
Improved Joint track information sharing to ensure interoperability of systems in theater operations
Coalition exchange and examination of real-time air picture data
Identification of Air Force Initiatives ready for warfighter transition
Investigated tactical UAS deployment in the National Airspace
Employment of Net-Enabled Weapons
JCAS immediate request amp end-to-end processes ldquoas-isrdquo characterization
Determined distributed system components were not ready for full live integration testing
Executed testing to support system-of-system interoperability certification
27
Customer Event Record Event Dates
Joint JIAMDO CorrelationDecorrelation Interoperability Test (CDIT)
Integration Events (Continuous) OCONUS October 2010
Navy UAS in NAS Runs for Record October 2010
Internal InterTEC Spiral 3 Systems Acceptance Test October - November 2010
Joint JITC Joint Interoperability Tests JIT 11-12345 (Continuous) October 2010 ndash September 2011
Air Force B1-B Fully Integrated Data Link Testing November 2010
Air Force AGILE Fire Phase III JEFX 2011 December 2010 - February 2011
Joint JTRS JPO -- JTRS Ground Mobile Radio January 2011
Navy Broad Area Maritime Surveillance (BAMS) (Continuous) January ndash September 2011
Joint Joint Track Manager Capability Demonstration (Continuous) January ndash September 2011
Joint JSF Initial MampS Interoperability (Continuous) February ndash March 2011
Air Force JSTARS Interoperability Test May 2011
Joint JS J8JIAMDO Joint Sensor Integration June ndash August 2011
FY11 JMETC Events
(More to Come)
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Global Hawk GATOR
Brigade Combat Team (BCT)
Modernization F-22 FY12 Testing Planned Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run
28
JMETC Connectivity
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca (2) JITC JTDL
Redstone (7) SMDC
RTC DTCC GMAN
SED Patriot THAAD FAAD SIV
Charleston (2)IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB SIMAF
As of 16 Mar 2011
0
10
20
30
40
50
60
70
FY06 FY07 FY08 FY09 FY10 FY11
JMETC Site Expansion
Sites
Dedicated trusted connectivity on SDREN (part of the GIG)
Encrypted for Secret ndash System High
DISA-registered IP address space
Active monitoring of network performance
Capable of supporting multiple simultaneous test events Bethpage NGC BAMS
Whiteman B-2
Eglin AFB (4)AOC DTF GWEF KHILS
Hurlburt Field C2DAC
Sites in Gulf Range
Functional Sites 61
New Sites Planned 8
Connection Points to Other Networks 5
Ft Greely CRTC
Site in Alaska
MHPCC
PMRF Bldg 105
Sites in Hawaii
Camp PendletonMCTSSA
China Lake (3)AV-8B FA-18IBAR
Edwards Ridley
Corona NSWC
El Segundo NGC B-2
Sites in SoCal
Point Loma (2)RLBTSSSC-PAC 59140 SSC-PAC HAIPE
Point Mugu (2)ITEC EA-6B
West Agg Rtr
Rancho Bernardo NGC BAMS
Pax River (5)ESTEL E2CD MCLACETEF SAIL ATR JMETC SYSCONEast Agg Rtr
Aberdeen ACCN
Langley (2) C-GIIF TDLITC
JFCOM JSIC
Wallops Island (2) SCSC SSDS
DISA Sky 7
Pentagon WARCAP
Dahlgren (2) CEDLIWSL
Dam Neck CDSA
Newport News NGC VASCIC
Sites in MD DC VA
Norfolk COMOPTEVFOR
McLean MITRE NCEL
Ft Huachuca JTRS
Bethpage NGC BCN
How an Air Force Customer Sees
Distributed Connectivity
Hardware-in-the-Loop (HWIL) Availability
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca JITC
Redstone (3) DTCC GMAN SED
Charleston (2) IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB
SIMAF
Bethpage NG BAMS
Whiteman B-2
MHPCC
PMRF Bldg 105
Sites in Hawaii
All linked by JMETC
29
Challenges Explained in Detail
30
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
Distributed Testing
A process preferably persistent and continuous for linking
various geographically separated Live Virtual and
Constructive sites and systems together in a distributed
environment for use across the acquisition life cycle to
support and conduct the Test and Evaluation (TampE) of a
system or systems-of-systems
GOAL Near Real-Time Test-Fix-Test
across organizational boundaries
3
Why Consider Distributed Test
bull Is there a requirement to exchange data within your system or within a
system-of-systems (SoS)
bull Do you have a requirement to address SoS interoperability issues early
in the acquisition process
bull Do you need more systems under test for live testing than what is
available at your facility
bull Do you have all of the appropriate ldquosupporting castrdquo (supporting
systems C4ISR assets etc) needed for testing
bull Do you need additional threat types fidelity and density in realistic
numbers at realistic ranges for testing
bull Do you need to incorporate TampE into training events
4
If you are answering lsquoNOrsquo to any of these
questions you will soon be saying lsquoYESrsquo
When Is Distributed Test Appropriate
bull Examples Where Appropriate
ndash Interoperability testing
bull C4 Interoperability with higher lower and adjacent Joint force organizations
ndash Data exchange in early DT testing
bull Interaction between sub-systems (latency may be a consideration)
bull Interaction between systems in a realistic environment
o Provide the most realistic environment possible from concept exploration through
Follow-On TampE
ndash When it is too costly to bring all the player systems together on a single
range
ndash Gain or increase operational relevance
bull Virtual and Constructive capabilities to supplement live systems (eg red
forces white forces terrain immobile test assets)
bull Examples Where Inappropriate
ndash System performance tests that do not include other systemssubsystems
ndash Reliability testing
5
Reduces Cost Risk and Time
6
JMETC Enabled Distributed Testing
Systems
Under
Test
Joint Operational Scenarios
Virtual
Prototype
Range Environment
Generator
Threat
Systems
Integrated Test
Resources
TENA Standard Interface
Definitions
TENA
Common
Middleware
TENA Standard Interface
Definitions
TENA
Common
Middleware
TENA Standard Interface
Definitions
TENA
Common
Middleware
TENA Standard Interface
Definitions
TENA
Common
Middleware
Reuse Repository
Distributed Test Support Tools
JMETC
Infrastructure
Hardware in the Loop Lab
TENA
Common
Middleware
Installed Systems
Test Facility
Basic Interface
Standard
TENA
Common
Middleware
Data Management Solutions
TENA
Standard Interface
Definitions
TENA
Standard Interface
Definitions
JMETC
Network on
SDREN
6
Distributed TampE Supports
Systems Engineering (SE)
Hardware Fabrication
ComponentInterface Definition
Allocated Functions amp Performance Requirements
Requirements Definitions (JCIDS) Transition
Requirements Analysis
Logical Analysis
Design Solution
Integration
Verification
Validation
Implementation
Tech Data amp Training Pubs
Software CreationCoding
Detailed Design
Confirmation Audits
Realization amp Assessment
Decomposition amp Definition
Input Output to Next Phase of
Development or Lifecycle Support Operational Need
Measures of EffectivenessSuitability
System Measures of Performance
Element Design Criteria
Spec
ialt
y En
gin
eeri
ng
Dev
elo
pm
ent
Des
ign
En
gin
eeri
ng
Dec
isio
n
Au
tho
rity
Tech Readiness
Assessment
Early TampE
Involvement
Component Level
Testing
Component
Characterization
Early Operational
Assessment
DTampE
Verification
OTampE
Validation
Source DAU Acquisition Community Connection
Early Injection of
Authoritative
systems and
mission threads
Combined DTampE
OTampE in a Joint
or SoS
Environment
Persistently Integrated
but disparately
developed capabilities
8
Sustainment Systems Acquisition (Engineering amp manufacturing development
demonstration LRIP amp production)
Pre-Systems Acquisition
IOC A B
Concept amp Tech Development
Concept
Exploration
Component
Advanced
Development
Decision
Review
System
Integration
Interim Progress Review
Production amp Deployment
LRIP
FRP Decision Review
Full-Rate Production amp Deployment
Operations
amp Support
C
The Future of Distributed Testing Test-Fix-Test
Across the Acquisition Life Cycle
Rapid Acquisition Developmental Test Operational
Test Interoperability Certification Net-Ready Key
Performance Parameters testing Joint Mission
Capability Portfolio testing
JMETC enables continuous testing across the acquisition life cycle JMETC reduces acquisition time and cost By Providing
bull Readily-available
persistent connectivity
bull Standing network
security agreements
bull Common
interoperability software
for linking sites
bull Certified test tools for
distributed testing
Enables early verification that
systems work stand alone and in a
Joint Environment
Helps find problems early in
acquisition ndash when they are less
costly to fix
Outline Distributed
Testing and JMETC
requirements in TEMP
Creates robust environment for common prototype analysis
Provides subject matter expertise to integrate distributed facilities
Test Early in Acquisition Process
Recent Distributed Test at a major test facility
bull C2 Interoperability upgrade to National Strategic asset
bull Significant software and interoperability problems found during initial Distributed Testing
bull Discovered system NOT ready for open air testing IMPACTmdashRisk Reduction Early
Identification of Problem
bullEarly Distributed Testing led to early
identification and correction of
interoperability issues
bullNo range or flying costs 9
Post-Milestone A ndash Concept Development
Joint Surface Warfare JCTD
Description
Point Mugu Test Team
demonstrated Net Enabled
Weapon Link-16 capability
using FA-18EF as launch
platform JSOW C-1 as
weapon and JSTARS as 3rd
party target source
Sample of Measures
Tactical message timing
between the controller
and the weapon
IMPACT-- Efficiency Lower Technical
Risk and Cost Savings
bull Program scheduled and executed short
multiple tests for incremental software
update evaluation
bull Resources expended on test amp analysis and
not network setup and monitoring
China Lake FA-18 JSOW
10
11
Description bull Assessment of B-2 Link-16
interoperability with AWACS
bull Connected live B-2 on ramp
at Whiteman AFB MO an
AWACS HITL at Tinker AFB
OK within a distributed C2
environment
IMPACT-- Cost Savings and Better
Product
bull Early testing led to early identification
and correction of Link 16 interoperability
issues
bull No range or flying costs
Army
Air Force
Navy
Marines
Joint
Industry
As of 14 Sep 2009
Tinker AFB AWACS
Whiteman B-2
Eglin 46TS
WPAFB SIMAF
JMETC
Langley GCIC
Army
Air Force
Navy
Marines
Joint
Industry
As of 14 Sep 2009
Tinker AFB AWACS
Whiteman B-2
Eglin 46TS
WPAFB SIMAF
JMETC
Langley GCIC
Post-Milestone C ndash Developmental Test
B-2 Spirit ICE Data Link Test
Sample of Measures
Timeliness and accuracy
of message exchanges
bull Joint Urgent Operational Need
bull Integration of BACN payload
onto multiple platforms for
solution to net-centric warfare
challenges of
bull Beyond line-of-sight comm
bull Relay bridge and range
extension for ground forces
and supporting aircraft
bull Distributed Testing included
Live-fly DT and Operational
Utility Evaluation Fall of 2010
IMPACT -- Cost Savings and
Better Product bullEfficient integration of DT and OT
bullAll required assets not available on-site
bullDistributed Testing saved ldquo$12Mrdquo
bullUrgent capability fielded
12
Responsive TampE for Rapid Fielding
Battlefield Airborne Communication Node
(BACN)
Challenges We Are Overcoming Today
Challenge
bull Site Interconnectivity
bull No common enabling
transport architecture
bull Inconsistent Tools
bull Test Participant
Collaboration
bull Analysis of Disparately
Stored Data
Overcome By
JMETC Network
TENA Middleware
JMETC ldquoBest of Breedrdquo
Process
Reuse Repository
Data Management Solutions
JMETC Joint Mission Environment Test Capability
TENA Test and Training Enabling Architecture
A TRMC Perspective
We Need a Common Vision for
Test and Training Infrastructure
Mr Derrick Hinton Principal Deputy Director Test Resource Management Center (TRMC)
Deputy Director Joint Investment Programs amp Policy (JIPP)
Office of the Undersecretary of Defense for Acquisition Technology amp Logistics (OUSD[ATampL])
Annual ITEA Test Instrumentation Workshop XIV
Test and Training Enterprise
12 May 2010
2010 2030
Individual Systems WHAT do we test Mission-Level
amp Systems of Systems Capabilities
Events WHEN do we test Continuously
Decision-Makers For WHOM do we test Decision-Makers
amp Warfighters
Agile
Streamlined
Affordable
Continuous
Goal Distributed Integrated Testing and Training
The DoD TampE process must evolve to be
Test Resource Management Center
Future TampE Process
15
Aspects of the 2030 Common
Vision
bull Test amp Training bull Real-Time TTP SOP Development to improve SoS effectiveness and
suitability (limited)
bull ldquoBlack boxrdquo and ldquowhite boxrdquo testing done in parallel
including
bull System functionality
bull System effectiveness
bull System of Systems effectiveness via mission threads in an
operationally relevant environment
bull Fewer test runsconfigurations with more meaningful
collected data
bull Real-Time Data Analysis amp Automated Reporting
OBJECTIVE Near Real-Time Test-Fix-Test 16
Some Challenges to Achieving
2030 Vision
bull Authoritative Joint Mission threads
bull Mission level measures
bull Thorough but Streamlined Information Assurance
Procedures
bull Enterprise-level Multi-Level Security solutions
bull Embedded Encryption on acquisition systems
bull RF over the wire (network)
bull Near Real Time Test-Fix-Test
bull Data Management
bull Embedded amp Unobtrusive Instrumentation
bull World wide wireless
bull Extremely efficient usage of the frequency spectrum 17
Examples Where the Vision Is Starting
to Come Together
bull Joint Integrated Air amp Missile Defense Organization
(JIAMDO) Correlation Decorrelation Interoperability Test
(CDIT) is reducing analysis cycle from 3 months to less
than a month
bull Changes to test configuration between test runs (~30 minutes)
bull Overnight AdditionSubtraction of test runs depending on issues
discovered and data analysis results
bull Real-time impact to test configuration and number of test runs
bull Emphasis on evaluators and tools for detailed data analysis
bull Air-Ground Integrated Layer Exploration (AGILE) bull AGILE provides an LVC capability for acquisition amp testing
bull Analytical Rigor provides decision quality data for decision support
bull Efficient amp collaborative use of Joint analytical resources (in
addition to service capabilities)
bull Sustains and evolves collaborative environment for future use 18
Some TRMC Investments for
Achieving Common Vision
bull Central TampE Investment Program (CTEIP)
bull Test and Training Enabling Architecture (TENA)
bull Multi-Level Security-Joint Coalition Network Environment (MLS-
JCNE)
bull InterTEC
bull Result JITC-certified instrumentation usable in all aspects of TampE
bull Common Range Integrated Instrumentation System (CRIIS)
bull TampE SampT Program
bull TENA in a Resource Constrained Environment (TRCE)
bull NST Evaluation Capability Model (NECM)
bull JMETC Program
bull DoD corporate approach for distributed testing 19
What Can You Do to Be Ready for
2030
bull Maintain Information Assurance agreements for all facility
systems
bull Streamline data collection amp analysis at your facility
bull Get comfortable with DoDAF views
bull Look across the RDTampE the community to find common
problems that can be addressed with similar solutions
bull TENA-enable your instrumentation systems and test
command amp control
bull Get involved share your requirements with your Service
representatives and in TRMC users meetings
20
JMETC Users Group
bull Purpose is to focus on technical requirements and solutions relevant to current and future Distributed Testing needs
bull Technical and Management level representatives identify core infrastructure requirements and most importantly resolve issues
bull Quarterly meetings of 250-300 JMETC customers acquisition programs test events ranges LVC sites tools and network providers
21
Last JMETC Users Group Meeting bull June 2011 bull Location Mid-East Coast bull Tracks
bull User Requirements bull Information Assurance
Security bull Data Management bull InterTEC (Current amp Planned) bull Networking
bull An established forum for the Distributed Test Community to
o Identify core infrastructure requirements and use cases
o Identify investigate amp resolve issues
o Identify opportunities to collaborate
o Discuss available solutions tools and techniques
o Share lessons learned
Summary
bull We are becoming more efficient in how we execute TampE
bull We must get the biggest and best ldquobang for the buckrdquo that we can
bull TRMC and Services continue TampE investments to enable realization of the 2030 Vision
bull Align TampE (OSD and Services) investments to mitigate test risk areas
bull The Future of TampE will include Distributed Integrated Testing and Training and will enable near real time Test-Fix-Test
22
JMETC Program Points of Contact
23
JMETC Program Manager Chip Ferguson chipfergusonosdmil 703-601-5274
JMETC Principal Deputy PM Bruce Bailey brucebaileyosdmil 703-601-5208
JMETC Lead Operations Planning Marty Arnwine martemasarnwineosdmil 703-601-5215
JMETC Senior Technical Advisor George Rumford georgerumfordosdmil
703-601-5233
JMETC Lead Systems Engineer Ryan Norman ryannormanosdmil
703-601-5277
JMETC Lead Network Engineer Arjuna ldquoAJrdquo Pathmanathan ArjunaPathmanathanosdmil
703-601-5214
JMETC Website wwwjmetcorg
Backup Slides
25
FY10 JMETC Events
Customer Event Record Event Dates
Air Force JEFX 10-1Spirit Ice (B2 Data Link Test) October ndash November 2009
Navy BAMS LVC DE October ndash December 2009
Air Force Battlefield Airborne Comm Node (BACN) JUON
(DTOT)
November 2009 to September 2010
Air Force Agile Fire 10-2 January 2010
Air Force JEFX 10-23 FebruaryApril 2010
Joint Joint Surface Warfare JCTD February to September 2010
JS J8JIAMDO Joint Sensor Integration April to September 2010
Air Force B1-B Fully Integrated Data Link Testing April 2010
JFCOM J8489 (TEST WEEK) JCAS Distributed Test June 2010
JIAMDO (Navy Lead) CorrelationDecorrelation Interoperability Test
(CDIT) Integration Events (Continuous)
July to September 2010
Army (Lead) UAS in the National Airspace July to September 2010
Air Force Agile Fire 10-3 August 2010
Joint JITC Joint Interoperability Tests JIT 10-3 amp 11-1
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Joint Strike Fighter (JSF) JIAMDOJoint Track Manager
Brigade Combat Team (BCT)
Modernization
Multi-Function Adv Data Link (MADL) Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run 25
26
JMETC FY10 Accomplishments
JMETC Accomplishments
Supported 88 distinct customer test activities
Expanded network from 38 to 57 sites
ATIN and JTDL Networks transitioned to JMETC
Upgraded JMETC support applications and utilities to TENA R6
DIACAP Tiger Team report completed and recommendations being executed
Enhanced JMETC services and capabilities provided by leveraging InterTEC Services and Industry
Reuse Repository usability improvements
FY10 Example JMETC Customers
Joint Integrated Air amp Missile Defense Organization (JIAMDO)
Broad Area Communications Node (BACN) JUON
B1-B
Broad Area Maritime Surveillance System (BAMS)
Air-Ground Integrated Layer Exploration (AGILE)
Joint Interoperability Test Command (JITC)
Selected Benefits to the DoD
Integrated DT amp OT on a Joint Urgent Operational Need for the warfighter
Maximized usage of theater assets during limited maintenance windows
Improved Joint track information sharing to ensure interoperability of systems in theater operations
Coalition exchange and examination of real-time air picture data
Identification of Air Force Initiatives ready for warfighter transition
Investigated tactical UAS deployment in the National Airspace
Employment of Net-Enabled Weapons
JCAS immediate request amp end-to-end processes ldquoas-isrdquo characterization
Determined distributed system components were not ready for full live integration testing
Executed testing to support system-of-system interoperability certification
27
Customer Event Record Event Dates
Joint JIAMDO CorrelationDecorrelation Interoperability Test (CDIT)
Integration Events (Continuous) OCONUS October 2010
Navy UAS in NAS Runs for Record October 2010
Internal InterTEC Spiral 3 Systems Acceptance Test October - November 2010
Joint JITC Joint Interoperability Tests JIT 11-12345 (Continuous) October 2010 ndash September 2011
Air Force B1-B Fully Integrated Data Link Testing November 2010
Air Force AGILE Fire Phase III JEFX 2011 December 2010 - February 2011
Joint JTRS JPO -- JTRS Ground Mobile Radio January 2011
Navy Broad Area Maritime Surveillance (BAMS) (Continuous) January ndash September 2011
Joint Joint Track Manager Capability Demonstration (Continuous) January ndash September 2011
Joint JSF Initial MampS Interoperability (Continuous) February ndash March 2011
Air Force JSTARS Interoperability Test May 2011
Joint JS J8JIAMDO Joint Sensor Integration June ndash August 2011
FY11 JMETC Events
(More to Come)
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Global Hawk GATOR
Brigade Combat Team (BCT)
Modernization F-22 FY12 Testing Planned Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run
28
JMETC Connectivity
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca (2) JITC JTDL
Redstone (7) SMDC
RTC DTCC GMAN
SED Patriot THAAD FAAD SIV
Charleston (2)IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB SIMAF
As of 16 Mar 2011
0
10
20
30
40
50
60
70
FY06 FY07 FY08 FY09 FY10 FY11
JMETC Site Expansion
Sites
Dedicated trusted connectivity on SDREN (part of the GIG)
Encrypted for Secret ndash System High
DISA-registered IP address space
Active monitoring of network performance
Capable of supporting multiple simultaneous test events Bethpage NGC BAMS
Whiteman B-2
Eglin AFB (4)AOC DTF GWEF KHILS
Hurlburt Field C2DAC
Sites in Gulf Range
Functional Sites 61
New Sites Planned 8
Connection Points to Other Networks 5
Ft Greely CRTC
Site in Alaska
MHPCC
PMRF Bldg 105
Sites in Hawaii
Camp PendletonMCTSSA
China Lake (3)AV-8B FA-18IBAR
Edwards Ridley
Corona NSWC
El Segundo NGC B-2
Sites in SoCal
Point Loma (2)RLBTSSSC-PAC 59140 SSC-PAC HAIPE
Point Mugu (2)ITEC EA-6B
West Agg Rtr
Rancho Bernardo NGC BAMS
Pax River (5)ESTEL E2CD MCLACETEF SAIL ATR JMETC SYSCONEast Agg Rtr
Aberdeen ACCN
Langley (2) C-GIIF TDLITC
JFCOM JSIC
Wallops Island (2) SCSC SSDS
DISA Sky 7
Pentagon WARCAP
Dahlgren (2) CEDLIWSL
Dam Neck CDSA
Newport News NGC VASCIC
Sites in MD DC VA
Norfolk COMOPTEVFOR
McLean MITRE NCEL
Ft Huachuca JTRS
Bethpage NGC BCN
How an Air Force Customer Sees
Distributed Connectivity
Hardware-in-the-Loop (HWIL) Availability
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca JITC
Redstone (3) DTCC GMAN SED
Charleston (2) IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB
SIMAF
Bethpage NG BAMS
Whiteman B-2
MHPCC
PMRF Bldg 105
Sites in Hawaii
All linked by JMETC
29
Challenges Explained in Detail
30
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
Why Consider Distributed Test
bull Is there a requirement to exchange data within your system or within a
system-of-systems (SoS)
bull Do you have a requirement to address SoS interoperability issues early
in the acquisition process
bull Do you need more systems under test for live testing than what is
available at your facility
bull Do you have all of the appropriate ldquosupporting castrdquo (supporting
systems C4ISR assets etc) needed for testing
bull Do you need additional threat types fidelity and density in realistic
numbers at realistic ranges for testing
bull Do you need to incorporate TampE into training events
4
If you are answering lsquoNOrsquo to any of these
questions you will soon be saying lsquoYESrsquo
When Is Distributed Test Appropriate
bull Examples Where Appropriate
ndash Interoperability testing
bull C4 Interoperability with higher lower and adjacent Joint force organizations
ndash Data exchange in early DT testing
bull Interaction between sub-systems (latency may be a consideration)
bull Interaction between systems in a realistic environment
o Provide the most realistic environment possible from concept exploration through
Follow-On TampE
ndash When it is too costly to bring all the player systems together on a single
range
ndash Gain or increase operational relevance
bull Virtual and Constructive capabilities to supplement live systems (eg red
forces white forces terrain immobile test assets)
bull Examples Where Inappropriate
ndash System performance tests that do not include other systemssubsystems
ndash Reliability testing
5
Reduces Cost Risk and Time
6
JMETC Enabled Distributed Testing
Systems
Under
Test
Joint Operational Scenarios
Virtual
Prototype
Range Environment
Generator
Threat
Systems
Integrated Test
Resources
TENA Standard Interface
Definitions
TENA
Common
Middleware
TENA Standard Interface
Definitions
TENA
Common
Middleware
TENA Standard Interface
Definitions
TENA
Common
Middleware
TENA Standard Interface
Definitions
TENA
Common
Middleware
Reuse Repository
Distributed Test Support Tools
JMETC
Infrastructure
Hardware in the Loop Lab
TENA
Common
Middleware
Installed Systems
Test Facility
Basic Interface
Standard
TENA
Common
Middleware
Data Management Solutions
TENA
Standard Interface
Definitions
TENA
Standard Interface
Definitions
JMETC
Network on
SDREN
6
Distributed TampE Supports
Systems Engineering (SE)
Hardware Fabrication
ComponentInterface Definition
Allocated Functions amp Performance Requirements
Requirements Definitions (JCIDS) Transition
Requirements Analysis
Logical Analysis
Design Solution
Integration
Verification
Validation
Implementation
Tech Data amp Training Pubs
Software CreationCoding
Detailed Design
Confirmation Audits
Realization amp Assessment
Decomposition amp Definition
Input Output to Next Phase of
Development or Lifecycle Support Operational Need
Measures of EffectivenessSuitability
System Measures of Performance
Element Design Criteria
Spec
ialt
y En
gin
eeri
ng
Dev
elo
pm
ent
Des
ign
En
gin
eeri
ng
Dec
isio
n
Au
tho
rity
Tech Readiness
Assessment
Early TampE
Involvement
Component Level
Testing
Component
Characterization
Early Operational
Assessment
DTampE
Verification
OTampE
Validation
Source DAU Acquisition Community Connection
Early Injection of
Authoritative
systems and
mission threads
Combined DTampE
OTampE in a Joint
or SoS
Environment
Persistently Integrated
but disparately
developed capabilities
8
Sustainment Systems Acquisition (Engineering amp manufacturing development
demonstration LRIP amp production)
Pre-Systems Acquisition
IOC A B
Concept amp Tech Development
Concept
Exploration
Component
Advanced
Development
Decision
Review
System
Integration
Interim Progress Review
Production amp Deployment
LRIP
FRP Decision Review
Full-Rate Production amp Deployment
Operations
amp Support
C
The Future of Distributed Testing Test-Fix-Test
Across the Acquisition Life Cycle
Rapid Acquisition Developmental Test Operational
Test Interoperability Certification Net-Ready Key
Performance Parameters testing Joint Mission
Capability Portfolio testing
JMETC enables continuous testing across the acquisition life cycle JMETC reduces acquisition time and cost By Providing
bull Readily-available
persistent connectivity
bull Standing network
security agreements
bull Common
interoperability software
for linking sites
bull Certified test tools for
distributed testing
Enables early verification that
systems work stand alone and in a
Joint Environment
Helps find problems early in
acquisition ndash when they are less
costly to fix
Outline Distributed
Testing and JMETC
requirements in TEMP
Creates robust environment for common prototype analysis
Provides subject matter expertise to integrate distributed facilities
Test Early in Acquisition Process
Recent Distributed Test at a major test facility
bull C2 Interoperability upgrade to National Strategic asset
bull Significant software and interoperability problems found during initial Distributed Testing
bull Discovered system NOT ready for open air testing IMPACTmdashRisk Reduction Early
Identification of Problem
bullEarly Distributed Testing led to early
identification and correction of
interoperability issues
bullNo range or flying costs 9
Post-Milestone A ndash Concept Development
Joint Surface Warfare JCTD
Description
Point Mugu Test Team
demonstrated Net Enabled
Weapon Link-16 capability
using FA-18EF as launch
platform JSOW C-1 as
weapon and JSTARS as 3rd
party target source
Sample of Measures
Tactical message timing
between the controller
and the weapon
IMPACT-- Efficiency Lower Technical
Risk and Cost Savings
bull Program scheduled and executed short
multiple tests for incremental software
update evaluation
bull Resources expended on test amp analysis and
not network setup and monitoring
China Lake FA-18 JSOW
10
11
Description bull Assessment of B-2 Link-16
interoperability with AWACS
bull Connected live B-2 on ramp
at Whiteman AFB MO an
AWACS HITL at Tinker AFB
OK within a distributed C2
environment
IMPACT-- Cost Savings and Better
Product
bull Early testing led to early identification
and correction of Link 16 interoperability
issues
bull No range or flying costs
Army
Air Force
Navy
Marines
Joint
Industry
As of 14 Sep 2009
Tinker AFB AWACS
Whiteman B-2
Eglin 46TS
WPAFB SIMAF
JMETC
Langley GCIC
Army
Air Force
Navy
Marines
Joint
Industry
As of 14 Sep 2009
Tinker AFB AWACS
Whiteman B-2
Eglin 46TS
WPAFB SIMAF
JMETC
Langley GCIC
Post-Milestone C ndash Developmental Test
B-2 Spirit ICE Data Link Test
Sample of Measures
Timeliness and accuracy
of message exchanges
bull Joint Urgent Operational Need
bull Integration of BACN payload
onto multiple platforms for
solution to net-centric warfare
challenges of
bull Beyond line-of-sight comm
bull Relay bridge and range
extension for ground forces
and supporting aircraft
bull Distributed Testing included
Live-fly DT and Operational
Utility Evaluation Fall of 2010
IMPACT -- Cost Savings and
Better Product bullEfficient integration of DT and OT
bullAll required assets not available on-site
bullDistributed Testing saved ldquo$12Mrdquo
bullUrgent capability fielded
12
Responsive TampE for Rapid Fielding
Battlefield Airborne Communication Node
(BACN)
Challenges We Are Overcoming Today
Challenge
bull Site Interconnectivity
bull No common enabling
transport architecture
bull Inconsistent Tools
bull Test Participant
Collaboration
bull Analysis of Disparately
Stored Data
Overcome By
JMETC Network
TENA Middleware
JMETC ldquoBest of Breedrdquo
Process
Reuse Repository
Data Management Solutions
JMETC Joint Mission Environment Test Capability
TENA Test and Training Enabling Architecture
A TRMC Perspective
We Need a Common Vision for
Test and Training Infrastructure
Mr Derrick Hinton Principal Deputy Director Test Resource Management Center (TRMC)
Deputy Director Joint Investment Programs amp Policy (JIPP)
Office of the Undersecretary of Defense for Acquisition Technology amp Logistics (OUSD[ATampL])
Annual ITEA Test Instrumentation Workshop XIV
Test and Training Enterprise
12 May 2010
2010 2030
Individual Systems WHAT do we test Mission-Level
amp Systems of Systems Capabilities
Events WHEN do we test Continuously
Decision-Makers For WHOM do we test Decision-Makers
amp Warfighters
Agile
Streamlined
Affordable
Continuous
Goal Distributed Integrated Testing and Training
The DoD TampE process must evolve to be
Test Resource Management Center
Future TampE Process
15
Aspects of the 2030 Common
Vision
bull Test amp Training bull Real-Time TTP SOP Development to improve SoS effectiveness and
suitability (limited)
bull ldquoBlack boxrdquo and ldquowhite boxrdquo testing done in parallel
including
bull System functionality
bull System effectiveness
bull System of Systems effectiveness via mission threads in an
operationally relevant environment
bull Fewer test runsconfigurations with more meaningful
collected data
bull Real-Time Data Analysis amp Automated Reporting
OBJECTIVE Near Real-Time Test-Fix-Test 16
Some Challenges to Achieving
2030 Vision
bull Authoritative Joint Mission threads
bull Mission level measures
bull Thorough but Streamlined Information Assurance
Procedures
bull Enterprise-level Multi-Level Security solutions
bull Embedded Encryption on acquisition systems
bull RF over the wire (network)
bull Near Real Time Test-Fix-Test
bull Data Management
bull Embedded amp Unobtrusive Instrumentation
bull World wide wireless
bull Extremely efficient usage of the frequency spectrum 17
Examples Where the Vision Is Starting
to Come Together
bull Joint Integrated Air amp Missile Defense Organization
(JIAMDO) Correlation Decorrelation Interoperability Test
(CDIT) is reducing analysis cycle from 3 months to less
than a month
bull Changes to test configuration between test runs (~30 minutes)
bull Overnight AdditionSubtraction of test runs depending on issues
discovered and data analysis results
bull Real-time impact to test configuration and number of test runs
bull Emphasis on evaluators and tools for detailed data analysis
bull Air-Ground Integrated Layer Exploration (AGILE) bull AGILE provides an LVC capability for acquisition amp testing
bull Analytical Rigor provides decision quality data for decision support
bull Efficient amp collaborative use of Joint analytical resources (in
addition to service capabilities)
bull Sustains and evolves collaborative environment for future use 18
Some TRMC Investments for
Achieving Common Vision
bull Central TampE Investment Program (CTEIP)
bull Test and Training Enabling Architecture (TENA)
bull Multi-Level Security-Joint Coalition Network Environment (MLS-
JCNE)
bull InterTEC
bull Result JITC-certified instrumentation usable in all aspects of TampE
bull Common Range Integrated Instrumentation System (CRIIS)
bull TampE SampT Program
bull TENA in a Resource Constrained Environment (TRCE)
bull NST Evaluation Capability Model (NECM)
bull JMETC Program
bull DoD corporate approach for distributed testing 19
What Can You Do to Be Ready for
2030
bull Maintain Information Assurance agreements for all facility
systems
bull Streamline data collection amp analysis at your facility
bull Get comfortable with DoDAF views
bull Look across the RDTampE the community to find common
problems that can be addressed with similar solutions
bull TENA-enable your instrumentation systems and test
command amp control
bull Get involved share your requirements with your Service
representatives and in TRMC users meetings
20
JMETC Users Group
bull Purpose is to focus on technical requirements and solutions relevant to current and future Distributed Testing needs
bull Technical and Management level representatives identify core infrastructure requirements and most importantly resolve issues
bull Quarterly meetings of 250-300 JMETC customers acquisition programs test events ranges LVC sites tools and network providers
21
Last JMETC Users Group Meeting bull June 2011 bull Location Mid-East Coast bull Tracks
bull User Requirements bull Information Assurance
Security bull Data Management bull InterTEC (Current amp Planned) bull Networking
bull An established forum for the Distributed Test Community to
o Identify core infrastructure requirements and use cases
o Identify investigate amp resolve issues
o Identify opportunities to collaborate
o Discuss available solutions tools and techniques
o Share lessons learned
Summary
bull We are becoming more efficient in how we execute TampE
bull We must get the biggest and best ldquobang for the buckrdquo that we can
bull TRMC and Services continue TampE investments to enable realization of the 2030 Vision
bull Align TampE (OSD and Services) investments to mitigate test risk areas
bull The Future of TampE will include Distributed Integrated Testing and Training and will enable near real time Test-Fix-Test
22
JMETC Program Points of Contact
23
JMETC Program Manager Chip Ferguson chipfergusonosdmil 703-601-5274
JMETC Principal Deputy PM Bruce Bailey brucebaileyosdmil 703-601-5208
JMETC Lead Operations Planning Marty Arnwine martemasarnwineosdmil 703-601-5215
JMETC Senior Technical Advisor George Rumford georgerumfordosdmil
703-601-5233
JMETC Lead Systems Engineer Ryan Norman ryannormanosdmil
703-601-5277
JMETC Lead Network Engineer Arjuna ldquoAJrdquo Pathmanathan ArjunaPathmanathanosdmil
703-601-5214
JMETC Website wwwjmetcorg
Backup Slides
25
FY10 JMETC Events
Customer Event Record Event Dates
Air Force JEFX 10-1Spirit Ice (B2 Data Link Test) October ndash November 2009
Navy BAMS LVC DE October ndash December 2009
Air Force Battlefield Airborne Comm Node (BACN) JUON
(DTOT)
November 2009 to September 2010
Air Force Agile Fire 10-2 January 2010
Air Force JEFX 10-23 FebruaryApril 2010
Joint Joint Surface Warfare JCTD February to September 2010
JS J8JIAMDO Joint Sensor Integration April to September 2010
Air Force B1-B Fully Integrated Data Link Testing April 2010
JFCOM J8489 (TEST WEEK) JCAS Distributed Test June 2010
JIAMDO (Navy Lead) CorrelationDecorrelation Interoperability Test
(CDIT) Integration Events (Continuous)
July to September 2010
Army (Lead) UAS in the National Airspace July to September 2010
Air Force Agile Fire 10-3 August 2010
Joint JITC Joint Interoperability Tests JIT 10-3 amp 11-1
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Joint Strike Fighter (JSF) JIAMDOJoint Track Manager
Brigade Combat Team (BCT)
Modernization
Multi-Function Adv Data Link (MADL) Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run 25
26
JMETC FY10 Accomplishments
JMETC Accomplishments
Supported 88 distinct customer test activities
Expanded network from 38 to 57 sites
ATIN and JTDL Networks transitioned to JMETC
Upgraded JMETC support applications and utilities to TENA R6
DIACAP Tiger Team report completed and recommendations being executed
Enhanced JMETC services and capabilities provided by leveraging InterTEC Services and Industry
Reuse Repository usability improvements
FY10 Example JMETC Customers
Joint Integrated Air amp Missile Defense Organization (JIAMDO)
Broad Area Communications Node (BACN) JUON
B1-B
Broad Area Maritime Surveillance System (BAMS)
Air-Ground Integrated Layer Exploration (AGILE)
Joint Interoperability Test Command (JITC)
Selected Benefits to the DoD
Integrated DT amp OT on a Joint Urgent Operational Need for the warfighter
Maximized usage of theater assets during limited maintenance windows
Improved Joint track information sharing to ensure interoperability of systems in theater operations
Coalition exchange and examination of real-time air picture data
Identification of Air Force Initiatives ready for warfighter transition
Investigated tactical UAS deployment in the National Airspace
Employment of Net-Enabled Weapons
JCAS immediate request amp end-to-end processes ldquoas-isrdquo characterization
Determined distributed system components were not ready for full live integration testing
Executed testing to support system-of-system interoperability certification
27
Customer Event Record Event Dates
Joint JIAMDO CorrelationDecorrelation Interoperability Test (CDIT)
Integration Events (Continuous) OCONUS October 2010
Navy UAS in NAS Runs for Record October 2010
Internal InterTEC Spiral 3 Systems Acceptance Test October - November 2010
Joint JITC Joint Interoperability Tests JIT 11-12345 (Continuous) October 2010 ndash September 2011
Air Force B1-B Fully Integrated Data Link Testing November 2010
Air Force AGILE Fire Phase III JEFX 2011 December 2010 - February 2011
Joint JTRS JPO -- JTRS Ground Mobile Radio January 2011
Navy Broad Area Maritime Surveillance (BAMS) (Continuous) January ndash September 2011
Joint Joint Track Manager Capability Demonstration (Continuous) January ndash September 2011
Joint JSF Initial MampS Interoperability (Continuous) February ndash March 2011
Air Force JSTARS Interoperability Test May 2011
Joint JS J8JIAMDO Joint Sensor Integration June ndash August 2011
FY11 JMETC Events
(More to Come)
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Global Hawk GATOR
Brigade Combat Team (BCT)
Modernization F-22 FY12 Testing Planned Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run
28
JMETC Connectivity
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca (2) JITC JTDL
Redstone (7) SMDC
RTC DTCC GMAN
SED Patriot THAAD FAAD SIV
Charleston (2)IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB SIMAF
As of 16 Mar 2011
0
10
20
30
40
50
60
70
FY06 FY07 FY08 FY09 FY10 FY11
JMETC Site Expansion
Sites
Dedicated trusted connectivity on SDREN (part of the GIG)
Encrypted for Secret ndash System High
DISA-registered IP address space
Active monitoring of network performance
Capable of supporting multiple simultaneous test events Bethpage NGC BAMS
Whiteman B-2
Eglin AFB (4)AOC DTF GWEF KHILS
Hurlburt Field C2DAC
Sites in Gulf Range
Functional Sites 61
New Sites Planned 8
Connection Points to Other Networks 5
Ft Greely CRTC
Site in Alaska
MHPCC
PMRF Bldg 105
Sites in Hawaii
Camp PendletonMCTSSA
China Lake (3)AV-8B FA-18IBAR
Edwards Ridley
Corona NSWC
El Segundo NGC B-2
Sites in SoCal
Point Loma (2)RLBTSSSC-PAC 59140 SSC-PAC HAIPE
Point Mugu (2)ITEC EA-6B
West Agg Rtr
Rancho Bernardo NGC BAMS
Pax River (5)ESTEL E2CD MCLACETEF SAIL ATR JMETC SYSCONEast Agg Rtr
Aberdeen ACCN
Langley (2) C-GIIF TDLITC
JFCOM JSIC
Wallops Island (2) SCSC SSDS
DISA Sky 7
Pentagon WARCAP
Dahlgren (2) CEDLIWSL
Dam Neck CDSA
Newport News NGC VASCIC
Sites in MD DC VA
Norfolk COMOPTEVFOR
McLean MITRE NCEL
Ft Huachuca JTRS
Bethpage NGC BCN
How an Air Force Customer Sees
Distributed Connectivity
Hardware-in-the-Loop (HWIL) Availability
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca JITC
Redstone (3) DTCC GMAN SED
Charleston (2) IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB
SIMAF
Bethpage NG BAMS
Whiteman B-2
MHPCC
PMRF Bldg 105
Sites in Hawaii
All linked by JMETC
29
Challenges Explained in Detail
30
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
When Is Distributed Test Appropriate
bull Examples Where Appropriate
ndash Interoperability testing
bull C4 Interoperability with higher lower and adjacent Joint force organizations
ndash Data exchange in early DT testing
bull Interaction between sub-systems (latency may be a consideration)
bull Interaction between systems in a realistic environment
o Provide the most realistic environment possible from concept exploration through
Follow-On TampE
ndash When it is too costly to bring all the player systems together on a single
range
ndash Gain or increase operational relevance
bull Virtual and Constructive capabilities to supplement live systems (eg red
forces white forces terrain immobile test assets)
bull Examples Where Inappropriate
ndash System performance tests that do not include other systemssubsystems
ndash Reliability testing
5
Reduces Cost Risk and Time
6
JMETC Enabled Distributed Testing
Systems
Under
Test
Joint Operational Scenarios
Virtual
Prototype
Range Environment
Generator
Threat
Systems
Integrated Test
Resources
TENA Standard Interface
Definitions
TENA
Common
Middleware
TENA Standard Interface
Definitions
TENA
Common
Middleware
TENA Standard Interface
Definitions
TENA
Common
Middleware
TENA Standard Interface
Definitions
TENA
Common
Middleware
Reuse Repository
Distributed Test Support Tools
JMETC
Infrastructure
Hardware in the Loop Lab
TENA
Common
Middleware
Installed Systems
Test Facility
Basic Interface
Standard
TENA
Common
Middleware
Data Management Solutions
TENA
Standard Interface
Definitions
TENA
Standard Interface
Definitions
JMETC
Network on
SDREN
6
Distributed TampE Supports
Systems Engineering (SE)
Hardware Fabrication
ComponentInterface Definition
Allocated Functions amp Performance Requirements
Requirements Definitions (JCIDS) Transition
Requirements Analysis
Logical Analysis
Design Solution
Integration
Verification
Validation
Implementation
Tech Data amp Training Pubs
Software CreationCoding
Detailed Design
Confirmation Audits
Realization amp Assessment
Decomposition amp Definition
Input Output to Next Phase of
Development or Lifecycle Support Operational Need
Measures of EffectivenessSuitability
System Measures of Performance
Element Design Criteria
Spec
ialt
y En
gin
eeri
ng
Dev
elo
pm
ent
Des
ign
En
gin
eeri
ng
Dec
isio
n
Au
tho
rity
Tech Readiness
Assessment
Early TampE
Involvement
Component Level
Testing
Component
Characterization
Early Operational
Assessment
DTampE
Verification
OTampE
Validation
Source DAU Acquisition Community Connection
Early Injection of
Authoritative
systems and
mission threads
Combined DTampE
OTampE in a Joint
or SoS
Environment
Persistently Integrated
but disparately
developed capabilities
8
Sustainment Systems Acquisition (Engineering amp manufacturing development
demonstration LRIP amp production)
Pre-Systems Acquisition
IOC A B
Concept amp Tech Development
Concept
Exploration
Component
Advanced
Development
Decision
Review
System
Integration
Interim Progress Review
Production amp Deployment
LRIP
FRP Decision Review
Full-Rate Production amp Deployment
Operations
amp Support
C
The Future of Distributed Testing Test-Fix-Test
Across the Acquisition Life Cycle
Rapid Acquisition Developmental Test Operational
Test Interoperability Certification Net-Ready Key
Performance Parameters testing Joint Mission
Capability Portfolio testing
JMETC enables continuous testing across the acquisition life cycle JMETC reduces acquisition time and cost By Providing
bull Readily-available
persistent connectivity
bull Standing network
security agreements
bull Common
interoperability software
for linking sites
bull Certified test tools for
distributed testing
Enables early verification that
systems work stand alone and in a
Joint Environment
Helps find problems early in
acquisition ndash when they are less
costly to fix
Outline Distributed
Testing and JMETC
requirements in TEMP
Creates robust environment for common prototype analysis
Provides subject matter expertise to integrate distributed facilities
Test Early in Acquisition Process
Recent Distributed Test at a major test facility
bull C2 Interoperability upgrade to National Strategic asset
bull Significant software and interoperability problems found during initial Distributed Testing
bull Discovered system NOT ready for open air testing IMPACTmdashRisk Reduction Early
Identification of Problem
bullEarly Distributed Testing led to early
identification and correction of
interoperability issues
bullNo range or flying costs 9
Post-Milestone A ndash Concept Development
Joint Surface Warfare JCTD
Description
Point Mugu Test Team
demonstrated Net Enabled
Weapon Link-16 capability
using FA-18EF as launch
platform JSOW C-1 as
weapon and JSTARS as 3rd
party target source
Sample of Measures
Tactical message timing
between the controller
and the weapon
IMPACT-- Efficiency Lower Technical
Risk and Cost Savings
bull Program scheduled and executed short
multiple tests for incremental software
update evaluation
bull Resources expended on test amp analysis and
not network setup and monitoring
China Lake FA-18 JSOW
10
11
Description bull Assessment of B-2 Link-16
interoperability with AWACS
bull Connected live B-2 on ramp
at Whiteman AFB MO an
AWACS HITL at Tinker AFB
OK within a distributed C2
environment
IMPACT-- Cost Savings and Better
Product
bull Early testing led to early identification
and correction of Link 16 interoperability
issues
bull No range or flying costs
Army
Air Force
Navy
Marines
Joint
Industry
As of 14 Sep 2009
Tinker AFB AWACS
Whiteman B-2
Eglin 46TS
WPAFB SIMAF
JMETC
Langley GCIC
Army
Air Force
Navy
Marines
Joint
Industry
As of 14 Sep 2009
Tinker AFB AWACS
Whiteman B-2
Eglin 46TS
WPAFB SIMAF
JMETC
Langley GCIC
Post-Milestone C ndash Developmental Test
B-2 Spirit ICE Data Link Test
Sample of Measures
Timeliness and accuracy
of message exchanges
bull Joint Urgent Operational Need
bull Integration of BACN payload
onto multiple platforms for
solution to net-centric warfare
challenges of
bull Beyond line-of-sight comm
bull Relay bridge and range
extension for ground forces
and supporting aircraft
bull Distributed Testing included
Live-fly DT and Operational
Utility Evaluation Fall of 2010
IMPACT -- Cost Savings and
Better Product bullEfficient integration of DT and OT
bullAll required assets not available on-site
bullDistributed Testing saved ldquo$12Mrdquo
bullUrgent capability fielded
12
Responsive TampE for Rapid Fielding
Battlefield Airborne Communication Node
(BACN)
Challenges We Are Overcoming Today
Challenge
bull Site Interconnectivity
bull No common enabling
transport architecture
bull Inconsistent Tools
bull Test Participant
Collaboration
bull Analysis of Disparately
Stored Data
Overcome By
JMETC Network
TENA Middleware
JMETC ldquoBest of Breedrdquo
Process
Reuse Repository
Data Management Solutions
JMETC Joint Mission Environment Test Capability
TENA Test and Training Enabling Architecture
A TRMC Perspective
We Need a Common Vision for
Test and Training Infrastructure
Mr Derrick Hinton Principal Deputy Director Test Resource Management Center (TRMC)
Deputy Director Joint Investment Programs amp Policy (JIPP)
Office of the Undersecretary of Defense for Acquisition Technology amp Logistics (OUSD[ATampL])
Annual ITEA Test Instrumentation Workshop XIV
Test and Training Enterprise
12 May 2010
2010 2030
Individual Systems WHAT do we test Mission-Level
amp Systems of Systems Capabilities
Events WHEN do we test Continuously
Decision-Makers For WHOM do we test Decision-Makers
amp Warfighters
Agile
Streamlined
Affordable
Continuous
Goal Distributed Integrated Testing and Training
The DoD TampE process must evolve to be
Test Resource Management Center
Future TampE Process
15
Aspects of the 2030 Common
Vision
bull Test amp Training bull Real-Time TTP SOP Development to improve SoS effectiveness and
suitability (limited)
bull ldquoBlack boxrdquo and ldquowhite boxrdquo testing done in parallel
including
bull System functionality
bull System effectiveness
bull System of Systems effectiveness via mission threads in an
operationally relevant environment
bull Fewer test runsconfigurations with more meaningful
collected data
bull Real-Time Data Analysis amp Automated Reporting
OBJECTIVE Near Real-Time Test-Fix-Test 16
Some Challenges to Achieving
2030 Vision
bull Authoritative Joint Mission threads
bull Mission level measures
bull Thorough but Streamlined Information Assurance
Procedures
bull Enterprise-level Multi-Level Security solutions
bull Embedded Encryption on acquisition systems
bull RF over the wire (network)
bull Near Real Time Test-Fix-Test
bull Data Management
bull Embedded amp Unobtrusive Instrumentation
bull World wide wireless
bull Extremely efficient usage of the frequency spectrum 17
Examples Where the Vision Is Starting
to Come Together
bull Joint Integrated Air amp Missile Defense Organization
(JIAMDO) Correlation Decorrelation Interoperability Test
(CDIT) is reducing analysis cycle from 3 months to less
than a month
bull Changes to test configuration between test runs (~30 minutes)
bull Overnight AdditionSubtraction of test runs depending on issues
discovered and data analysis results
bull Real-time impact to test configuration and number of test runs
bull Emphasis on evaluators and tools for detailed data analysis
bull Air-Ground Integrated Layer Exploration (AGILE) bull AGILE provides an LVC capability for acquisition amp testing
bull Analytical Rigor provides decision quality data for decision support
bull Efficient amp collaborative use of Joint analytical resources (in
addition to service capabilities)
bull Sustains and evolves collaborative environment for future use 18
Some TRMC Investments for
Achieving Common Vision
bull Central TampE Investment Program (CTEIP)
bull Test and Training Enabling Architecture (TENA)
bull Multi-Level Security-Joint Coalition Network Environment (MLS-
JCNE)
bull InterTEC
bull Result JITC-certified instrumentation usable in all aspects of TampE
bull Common Range Integrated Instrumentation System (CRIIS)
bull TampE SampT Program
bull TENA in a Resource Constrained Environment (TRCE)
bull NST Evaluation Capability Model (NECM)
bull JMETC Program
bull DoD corporate approach for distributed testing 19
What Can You Do to Be Ready for
2030
bull Maintain Information Assurance agreements for all facility
systems
bull Streamline data collection amp analysis at your facility
bull Get comfortable with DoDAF views
bull Look across the RDTampE the community to find common
problems that can be addressed with similar solutions
bull TENA-enable your instrumentation systems and test
command amp control
bull Get involved share your requirements with your Service
representatives and in TRMC users meetings
20
JMETC Users Group
bull Purpose is to focus on technical requirements and solutions relevant to current and future Distributed Testing needs
bull Technical and Management level representatives identify core infrastructure requirements and most importantly resolve issues
bull Quarterly meetings of 250-300 JMETC customers acquisition programs test events ranges LVC sites tools and network providers
21
Last JMETC Users Group Meeting bull June 2011 bull Location Mid-East Coast bull Tracks
bull User Requirements bull Information Assurance
Security bull Data Management bull InterTEC (Current amp Planned) bull Networking
bull An established forum for the Distributed Test Community to
o Identify core infrastructure requirements and use cases
o Identify investigate amp resolve issues
o Identify opportunities to collaborate
o Discuss available solutions tools and techniques
o Share lessons learned
Summary
bull We are becoming more efficient in how we execute TampE
bull We must get the biggest and best ldquobang for the buckrdquo that we can
bull TRMC and Services continue TampE investments to enable realization of the 2030 Vision
bull Align TampE (OSD and Services) investments to mitigate test risk areas
bull The Future of TampE will include Distributed Integrated Testing and Training and will enable near real time Test-Fix-Test
22
JMETC Program Points of Contact
23
JMETC Program Manager Chip Ferguson chipfergusonosdmil 703-601-5274
JMETC Principal Deputy PM Bruce Bailey brucebaileyosdmil 703-601-5208
JMETC Lead Operations Planning Marty Arnwine martemasarnwineosdmil 703-601-5215
JMETC Senior Technical Advisor George Rumford georgerumfordosdmil
703-601-5233
JMETC Lead Systems Engineer Ryan Norman ryannormanosdmil
703-601-5277
JMETC Lead Network Engineer Arjuna ldquoAJrdquo Pathmanathan ArjunaPathmanathanosdmil
703-601-5214
JMETC Website wwwjmetcorg
Backup Slides
25
FY10 JMETC Events
Customer Event Record Event Dates
Air Force JEFX 10-1Spirit Ice (B2 Data Link Test) October ndash November 2009
Navy BAMS LVC DE October ndash December 2009
Air Force Battlefield Airborne Comm Node (BACN) JUON
(DTOT)
November 2009 to September 2010
Air Force Agile Fire 10-2 January 2010
Air Force JEFX 10-23 FebruaryApril 2010
Joint Joint Surface Warfare JCTD February to September 2010
JS J8JIAMDO Joint Sensor Integration April to September 2010
Air Force B1-B Fully Integrated Data Link Testing April 2010
JFCOM J8489 (TEST WEEK) JCAS Distributed Test June 2010
JIAMDO (Navy Lead) CorrelationDecorrelation Interoperability Test
(CDIT) Integration Events (Continuous)
July to September 2010
Army (Lead) UAS in the National Airspace July to September 2010
Air Force Agile Fire 10-3 August 2010
Joint JITC Joint Interoperability Tests JIT 10-3 amp 11-1
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Joint Strike Fighter (JSF) JIAMDOJoint Track Manager
Brigade Combat Team (BCT)
Modernization
Multi-Function Adv Data Link (MADL) Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run 25
26
JMETC FY10 Accomplishments
JMETC Accomplishments
Supported 88 distinct customer test activities
Expanded network from 38 to 57 sites
ATIN and JTDL Networks transitioned to JMETC
Upgraded JMETC support applications and utilities to TENA R6
DIACAP Tiger Team report completed and recommendations being executed
Enhanced JMETC services and capabilities provided by leveraging InterTEC Services and Industry
Reuse Repository usability improvements
FY10 Example JMETC Customers
Joint Integrated Air amp Missile Defense Organization (JIAMDO)
Broad Area Communications Node (BACN) JUON
B1-B
Broad Area Maritime Surveillance System (BAMS)
Air-Ground Integrated Layer Exploration (AGILE)
Joint Interoperability Test Command (JITC)
Selected Benefits to the DoD
Integrated DT amp OT on a Joint Urgent Operational Need for the warfighter
Maximized usage of theater assets during limited maintenance windows
Improved Joint track information sharing to ensure interoperability of systems in theater operations
Coalition exchange and examination of real-time air picture data
Identification of Air Force Initiatives ready for warfighter transition
Investigated tactical UAS deployment in the National Airspace
Employment of Net-Enabled Weapons
JCAS immediate request amp end-to-end processes ldquoas-isrdquo characterization
Determined distributed system components were not ready for full live integration testing
Executed testing to support system-of-system interoperability certification
27
Customer Event Record Event Dates
Joint JIAMDO CorrelationDecorrelation Interoperability Test (CDIT)
Integration Events (Continuous) OCONUS October 2010
Navy UAS in NAS Runs for Record October 2010
Internal InterTEC Spiral 3 Systems Acceptance Test October - November 2010
Joint JITC Joint Interoperability Tests JIT 11-12345 (Continuous) October 2010 ndash September 2011
Air Force B1-B Fully Integrated Data Link Testing November 2010
Air Force AGILE Fire Phase III JEFX 2011 December 2010 - February 2011
Joint JTRS JPO -- JTRS Ground Mobile Radio January 2011
Navy Broad Area Maritime Surveillance (BAMS) (Continuous) January ndash September 2011
Joint Joint Track Manager Capability Demonstration (Continuous) January ndash September 2011
Joint JSF Initial MampS Interoperability (Continuous) February ndash March 2011
Air Force JSTARS Interoperability Test May 2011
Joint JS J8JIAMDO Joint Sensor Integration June ndash August 2011
FY11 JMETC Events
(More to Come)
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Global Hawk GATOR
Brigade Combat Team (BCT)
Modernization F-22 FY12 Testing Planned Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run
28
JMETC Connectivity
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca (2) JITC JTDL
Redstone (7) SMDC
RTC DTCC GMAN
SED Patriot THAAD FAAD SIV
Charleston (2)IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB SIMAF
As of 16 Mar 2011
0
10
20
30
40
50
60
70
FY06 FY07 FY08 FY09 FY10 FY11
JMETC Site Expansion
Sites
Dedicated trusted connectivity on SDREN (part of the GIG)
Encrypted for Secret ndash System High
DISA-registered IP address space
Active monitoring of network performance
Capable of supporting multiple simultaneous test events Bethpage NGC BAMS
Whiteman B-2
Eglin AFB (4)AOC DTF GWEF KHILS
Hurlburt Field C2DAC
Sites in Gulf Range
Functional Sites 61
New Sites Planned 8
Connection Points to Other Networks 5
Ft Greely CRTC
Site in Alaska
MHPCC
PMRF Bldg 105
Sites in Hawaii
Camp PendletonMCTSSA
China Lake (3)AV-8B FA-18IBAR
Edwards Ridley
Corona NSWC
El Segundo NGC B-2
Sites in SoCal
Point Loma (2)RLBTSSSC-PAC 59140 SSC-PAC HAIPE
Point Mugu (2)ITEC EA-6B
West Agg Rtr
Rancho Bernardo NGC BAMS
Pax River (5)ESTEL E2CD MCLACETEF SAIL ATR JMETC SYSCONEast Agg Rtr
Aberdeen ACCN
Langley (2) C-GIIF TDLITC
JFCOM JSIC
Wallops Island (2) SCSC SSDS
DISA Sky 7
Pentagon WARCAP
Dahlgren (2) CEDLIWSL
Dam Neck CDSA
Newport News NGC VASCIC
Sites in MD DC VA
Norfolk COMOPTEVFOR
McLean MITRE NCEL
Ft Huachuca JTRS
Bethpage NGC BCN
How an Air Force Customer Sees
Distributed Connectivity
Hardware-in-the-Loop (HWIL) Availability
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca JITC
Redstone (3) DTCC GMAN SED
Charleston (2) IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB
SIMAF
Bethpage NG BAMS
Whiteman B-2
MHPCC
PMRF Bldg 105
Sites in Hawaii
All linked by JMETC
29
Challenges Explained in Detail
30
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
6
JMETC Enabled Distributed Testing
Systems
Under
Test
Joint Operational Scenarios
Virtual
Prototype
Range Environment
Generator
Threat
Systems
Integrated Test
Resources
TENA Standard Interface
Definitions
TENA
Common
Middleware
TENA Standard Interface
Definitions
TENA
Common
Middleware
TENA Standard Interface
Definitions
TENA
Common
Middleware
TENA Standard Interface
Definitions
TENA
Common
Middleware
Reuse Repository
Distributed Test Support Tools
JMETC
Infrastructure
Hardware in the Loop Lab
TENA
Common
Middleware
Installed Systems
Test Facility
Basic Interface
Standard
TENA
Common
Middleware
Data Management Solutions
TENA
Standard Interface
Definitions
TENA
Standard Interface
Definitions
JMETC
Network on
SDREN
6
Distributed TampE Supports
Systems Engineering (SE)
Hardware Fabrication
ComponentInterface Definition
Allocated Functions amp Performance Requirements
Requirements Definitions (JCIDS) Transition
Requirements Analysis
Logical Analysis
Design Solution
Integration
Verification
Validation
Implementation
Tech Data amp Training Pubs
Software CreationCoding
Detailed Design
Confirmation Audits
Realization amp Assessment
Decomposition amp Definition
Input Output to Next Phase of
Development or Lifecycle Support Operational Need
Measures of EffectivenessSuitability
System Measures of Performance
Element Design Criteria
Spec
ialt
y En
gin
eeri
ng
Dev
elo
pm
ent
Des
ign
En
gin
eeri
ng
Dec
isio
n
Au
tho
rity
Tech Readiness
Assessment
Early TampE
Involvement
Component Level
Testing
Component
Characterization
Early Operational
Assessment
DTampE
Verification
OTampE
Validation
Source DAU Acquisition Community Connection
Early Injection of
Authoritative
systems and
mission threads
Combined DTampE
OTampE in a Joint
or SoS
Environment
Persistently Integrated
but disparately
developed capabilities
8
Sustainment Systems Acquisition (Engineering amp manufacturing development
demonstration LRIP amp production)
Pre-Systems Acquisition
IOC A B
Concept amp Tech Development
Concept
Exploration
Component
Advanced
Development
Decision
Review
System
Integration
Interim Progress Review
Production amp Deployment
LRIP
FRP Decision Review
Full-Rate Production amp Deployment
Operations
amp Support
C
The Future of Distributed Testing Test-Fix-Test
Across the Acquisition Life Cycle
Rapid Acquisition Developmental Test Operational
Test Interoperability Certification Net-Ready Key
Performance Parameters testing Joint Mission
Capability Portfolio testing
JMETC enables continuous testing across the acquisition life cycle JMETC reduces acquisition time and cost By Providing
bull Readily-available
persistent connectivity
bull Standing network
security agreements
bull Common
interoperability software
for linking sites
bull Certified test tools for
distributed testing
Enables early verification that
systems work stand alone and in a
Joint Environment
Helps find problems early in
acquisition ndash when they are less
costly to fix
Outline Distributed
Testing and JMETC
requirements in TEMP
Creates robust environment for common prototype analysis
Provides subject matter expertise to integrate distributed facilities
Test Early in Acquisition Process
Recent Distributed Test at a major test facility
bull C2 Interoperability upgrade to National Strategic asset
bull Significant software and interoperability problems found during initial Distributed Testing
bull Discovered system NOT ready for open air testing IMPACTmdashRisk Reduction Early
Identification of Problem
bullEarly Distributed Testing led to early
identification and correction of
interoperability issues
bullNo range or flying costs 9
Post-Milestone A ndash Concept Development
Joint Surface Warfare JCTD
Description
Point Mugu Test Team
demonstrated Net Enabled
Weapon Link-16 capability
using FA-18EF as launch
platform JSOW C-1 as
weapon and JSTARS as 3rd
party target source
Sample of Measures
Tactical message timing
between the controller
and the weapon
IMPACT-- Efficiency Lower Technical
Risk and Cost Savings
bull Program scheduled and executed short
multiple tests for incremental software
update evaluation
bull Resources expended on test amp analysis and
not network setup and monitoring
China Lake FA-18 JSOW
10
11
Description bull Assessment of B-2 Link-16
interoperability with AWACS
bull Connected live B-2 on ramp
at Whiteman AFB MO an
AWACS HITL at Tinker AFB
OK within a distributed C2
environment
IMPACT-- Cost Savings and Better
Product
bull Early testing led to early identification
and correction of Link 16 interoperability
issues
bull No range or flying costs
Army
Air Force
Navy
Marines
Joint
Industry
As of 14 Sep 2009
Tinker AFB AWACS
Whiteman B-2
Eglin 46TS
WPAFB SIMAF
JMETC
Langley GCIC
Army
Air Force
Navy
Marines
Joint
Industry
As of 14 Sep 2009
Tinker AFB AWACS
Whiteman B-2
Eglin 46TS
WPAFB SIMAF
JMETC
Langley GCIC
Post-Milestone C ndash Developmental Test
B-2 Spirit ICE Data Link Test
Sample of Measures
Timeliness and accuracy
of message exchanges
bull Joint Urgent Operational Need
bull Integration of BACN payload
onto multiple platforms for
solution to net-centric warfare
challenges of
bull Beyond line-of-sight comm
bull Relay bridge and range
extension for ground forces
and supporting aircraft
bull Distributed Testing included
Live-fly DT and Operational
Utility Evaluation Fall of 2010
IMPACT -- Cost Savings and
Better Product bullEfficient integration of DT and OT
bullAll required assets not available on-site
bullDistributed Testing saved ldquo$12Mrdquo
bullUrgent capability fielded
12
Responsive TampE for Rapid Fielding
Battlefield Airborne Communication Node
(BACN)
Challenges We Are Overcoming Today
Challenge
bull Site Interconnectivity
bull No common enabling
transport architecture
bull Inconsistent Tools
bull Test Participant
Collaboration
bull Analysis of Disparately
Stored Data
Overcome By
JMETC Network
TENA Middleware
JMETC ldquoBest of Breedrdquo
Process
Reuse Repository
Data Management Solutions
JMETC Joint Mission Environment Test Capability
TENA Test and Training Enabling Architecture
A TRMC Perspective
We Need a Common Vision for
Test and Training Infrastructure
Mr Derrick Hinton Principal Deputy Director Test Resource Management Center (TRMC)
Deputy Director Joint Investment Programs amp Policy (JIPP)
Office of the Undersecretary of Defense for Acquisition Technology amp Logistics (OUSD[ATampL])
Annual ITEA Test Instrumentation Workshop XIV
Test and Training Enterprise
12 May 2010
2010 2030
Individual Systems WHAT do we test Mission-Level
amp Systems of Systems Capabilities
Events WHEN do we test Continuously
Decision-Makers For WHOM do we test Decision-Makers
amp Warfighters
Agile
Streamlined
Affordable
Continuous
Goal Distributed Integrated Testing and Training
The DoD TampE process must evolve to be
Test Resource Management Center
Future TampE Process
15
Aspects of the 2030 Common
Vision
bull Test amp Training bull Real-Time TTP SOP Development to improve SoS effectiveness and
suitability (limited)
bull ldquoBlack boxrdquo and ldquowhite boxrdquo testing done in parallel
including
bull System functionality
bull System effectiveness
bull System of Systems effectiveness via mission threads in an
operationally relevant environment
bull Fewer test runsconfigurations with more meaningful
collected data
bull Real-Time Data Analysis amp Automated Reporting
OBJECTIVE Near Real-Time Test-Fix-Test 16
Some Challenges to Achieving
2030 Vision
bull Authoritative Joint Mission threads
bull Mission level measures
bull Thorough but Streamlined Information Assurance
Procedures
bull Enterprise-level Multi-Level Security solutions
bull Embedded Encryption on acquisition systems
bull RF over the wire (network)
bull Near Real Time Test-Fix-Test
bull Data Management
bull Embedded amp Unobtrusive Instrumentation
bull World wide wireless
bull Extremely efficient usage of the frequency spectrum 17
Examples Where the Vision Is Starting
to Come Together
bull Joint Integrated Air amp Missile Defense Organization
(JIAMDO) Correlation Decorrelation Interoperability Test
(CDIT) is reducing analysis cycle from 3 months to less
than a month
bull Changes to test configuration between test runs (~30 minutes)
bull Overnight AdditionSubtraction of test runs depending on issues
discovered and data analysis results
bull Real-time impact to test configuration and number of test runs
bull Emphasis on evaluators and tools for detailed data analysis
bull Air-Ground Integrated Layer Exploration (AGILE) bull AGILE provides an LVC capability for acquisition amp testing
bull Analytical Rigor provides decision quality data for decision support
bull Efficient amp collaborative use of Joint analytical resources (in
addition to service capabilities)
bull Sustains and evolves collaborative environment for future use 18
Some TRMC Investments for
Achieving Common Vision
bull Central TampE Investment Program (CTEIP)
bull Test and Training Enabling Architecture (TENA)
bull Multi-Level Security-Joint Coalition Network Environment (MLS-
JCNE)
bull InterTEC
bull Result JITC-certified instrumentation usable in all aspects of TampE
bull Common Range Integrated Instrumentation System (CRIIS)
bull TampE SampT Program
bull TENA in a Resource Constrained Environment (TRCE)
bull NST Evaluation Capability Model (NECM)
bull JMETC Program
bull DoD corporate approach for distributed testing 19
What Can You Do to Be Ready for
2030
bull Maintain Information Assurance agreements for all facility
systems
bull Streamline data collection amp analysis at your facility
bull Get comfortable with DoDAF views
bull Look across the RDTampE the community to find common
problems that can be addressed with similar solutions
bull TENA-enable your instrumentation systems and test
command amp control
bull Get involved share your requirements with your Service
representatives and in TRMC users meetings
20
JMETC Users Group
bull Purpose is to focus on technical requirements and solutions relevant to current and future Distributed Testing needs
bull Technical and Management level representatives identify core infrastructure requirements and most importantly resolve issues
bull Quarterly meetings of 250-300 JMETC customers acquisition programs test events ranges LVC sites tools and network providers
21
Last JMETC Users Group Meeting bull June 2011 bull Location Mid-East Coast bull Tracks
bull User Requirements bull Information Assurance
Security bull Data Management bull InterTEC (Current amp Planned) bull Networking
bull An established forum for the Distributed Test Community to
o Identify core infrastructure requirements and use cases
o Identify investigate amp resolve issues
o Identify opportunities to collaborate
o Discuss available solutions tools and techniques
o Share lessons learned
Summary
bull We are becoming more efficient in how we execute TampE
bull We must get the biggest and best ldquobang for the buckrdquo that we can
bull TRMC and Services continue TampE investments to enable realization of the 2030 Vision
bull Align TampE (OSD and Services) investments to mitigate test risk areas
bull The Future of TampE will include Distributed Integrated Testing and Training and will enable near real time Test-Fix-Test
22
JMETC Program Points of Contact
23
JMETC Program Manager Chip Ferguson chipfergusonosdmil 703-601-5274
JMETC Principal Deputy PM Bruce Bailey brucebaileyosdmil 703-601-5208
JMETC Lead Operations Planning Marty Arnwine martemasarnwineosdmil 703-601-5215
JMETC Senior Technical Advisor George Rumford georgerumfordosdmil
703-601-5233
JMETC Lead Systems Engineer Ryan Norman ryannormanosdmil
703-601-5277
JMETC Lead Network Engineer Arjuna ldquoAJrdquo Pathmanathan ArjunaPathmanathanosdmil
703-601-5214
JMETC Website wwwjmetcorg
Backup Slides
25
FY10 JMETC Events
Customer Event Record Event Dates
Air Force JEFX 10-1Spirit Ice (B2 Data Link Test) October ndash November 2009
Navy BAMS LVC DE October ndash December 2009
Air Force Battlefield Airborne Comm Node (BACN) JUON
(DTOT)
November 2009 to September 2010
Air Force Agile Fire 10-2 January 2010
Air Force JEFX 10-23 FebruaryApril 2010
Joint Joint Surface Warfare JCTD February to September 2010
JS J8JIAMDO Joint Sensor Integration April to September 2010
Air Force B1-B Fully Integrated Data Link Testing April 2010
JFCOM J8489 (TEST WEEK) JCAS Distributed Test June 2010
JIAMDO (Navy Lead) CorrelationDecorrelation Interoperability Test
(CDIT) Integration Events (Continuous)
July to September 2010
Army (Lead) UAS in the National Airspace July to September 2010
Air Force Agile Fire 10-3 August 2010
Joint JITC Joint Interoperability Tests JIT 10-3 amp 11-1
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Joint Strike Fighter (JSF) JIAMDOJoint Track Manager
Brigade Combat Team (BCT)
Modernization
Multi-Function Adv Data Link (MADL) Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run 25
26
JMETC FY10 Accomplishments
JMETC Accomplishments
Supported 88 distinct customer test activities
Expanded network from 38 to 57 sites
ATIN and JTDL Networks transitioned to JMETC
Upgraded JMETC support applications and utilities to TENA R6
DIACAP Tiger Team report completed and recommendations being executed
Enhanced JMETC services and capabilities provided by leveraging InterTEC Services and Industry
Reuse Repository usability improvements
FY10 Example JMETC Customers
Joint Integrated Air amp Missile Defense Organization (JIAMDO)
Broad Area Communications Node (BACN) JUON
B1-B
Broad Area Maritime Surveillance System (BAMS)
Air-Ground Integrated Layer Exploration (AGILE)
Joint Interoperability Test Command (JITC)
Selected Benefits to the DoD
Integrated DT amp OT on a Joint Urgent Operational Need for the warfighter
Maximized usage of theater assets during limited maintenance windows
Improved Joint track information sharing to ensure interoperability of systems in theater operations
Coalition exchange and examination of real-time air picture data
Identification of Air Force Initiatives ready for warfighter transition
Investigated tactical UAS deployment in the National Airspace
Employment of Net-Enabled Weapons
JCAS immediate request amp end-to-end processes ldquoas-isrdquo characterization
Determined distributed system components were not ready for full live integration testing
Executed testing to support system-of-system interoperability certification
27
Customer Event Record Event Dates
Joint JIAMDO CorrelationDecorrelation Interoperability Test (CDIT)
Integration Events (Continuous) OCONUS October 2010
Navy UAS in NAS Runs for Record October 2010
Internal InterTEC Spiral 3 Systems Acceptance Test October - November 2010
Joint JITC Joint Interoperability Tests JIT 11-12345 (Continuous) October 2010 ndash September 2011
Air Force B1-B Fully Integrated Data Link Testing November 2010
Air Force AGILE Fire Phase III JEFX 2011 December 2010 - February 2011
Joint JTRS JPO -- JTRS Ground Mobile Radio January 2011
Navy Broad Area Maritime Surveillance (BAMS) (Continuous) January ndash September 2011
Joint Joint Track Manager Capability Demonstration (Continuous) January ndash September 2011
Joint JSF Initial MampS Interoperability (Continuous) February ndash March 2011
Air Force JSTARS Interoperability Test May 2011
Joint JS J8JIAMDO Joint Sensor Integration June ndash August 2011
FY11 JMETC Events
(More to Come)
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Global Hawk GATOR
Brigade Combat Team (BCT)
Modernization F-22 FY12 Testing Planned Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run
28
JMETC Connectivity
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca (2) JITC JTDL
Redstone (7) SMDC
RTC DTCC GMAN
SED Patriot THAAD FAAD SIV
Charleston (2)IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB SIMAF
As of 16 Mar 2011
0
10
20
30
40
50
60
70
FY06 FY07 FY08 FY09 FY10 FY11
JMETC Site Expansion
Sites
Dedicated trusted connectivity on SDREN (part of the GIG)
Encrypted for Secret ndash System High
DISA-registered IP address space
Active monitoring of network performance
Capable of supporting multiple simultaneous test events Bethpage NGC BAMS
Whiteman B-2
Eglin AFB (4)AOC DTF GWEF KHILS
Hurlburt Field C2DAC
Sites in Gulf Range
Functional Sites 61
New Sites Planned 8
Connection Points to Other Networks 5
Ft Greely CRTC
Site in Alaska
MHPCC
PMRF Bldg 105
Sites in Hawaii
Camp PendletonMCTSSA
China Lake (3)AV-8B FA-18IBAR
Edwards Ridley
Corona NSWC
El Segundo NGC B-2
Sites in SoCal
Point Loma (2)RLBTSSSC-PAC 59140 SSC-PAC HAIPE
Point Mugu (2)ITEC EA-6B
West Agg Rtr
Rancho Bernardo NGC BAMS
Pax River (5)ESTEL E2CD MCLACETEF SAIL ATR JMETC SYSCONEast Agg Rtr
Aberdeen ACCN
Langley (2) C-GIIF TDLITC
JFCOM JSIC
Wallops Island (2) SCSC SSDS
DISA Sky 7
Pentagon WARCAP
Dahlgren (2) CEDLIWSL
Dam Neck CDSA
Newport News NGC VASCIC
Sites in MD DC VA
Norfolk COMOPTEVFOR
McLean MITRE NCEL
Ft Huachuca JTRS
Bethpage NGC BCN
How an Air Force Customer Sees
Distributed Connectivity
Hardware-in-the-Loop (HWIL) Availability
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca JITC
Redstone (3) DTCC GMAN SED
Charleston (2) IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB
SIMAF
Bethpage NG BAMS
Whiteman B-2
MHPCC
PMRF Bldg 105
Sites in Hawaii
All linked by JMETC
29
Challenges Explained in Detail
30
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
Distributed TampE Supports
Systems Engineering (SE)
Hardware Fabrication
ComponentInterface Definition
Allocated Functions amp Performance Requirements
Requirements Definitions (JCIDS) Transition
Requirements Analysis
Logical Analysis
Design Solution
Integration
Verification
Validation
Implementation
Tech Data amp Training Pubs
Software CreationCoding
Detailed Design
Confirmation Audits
Realization amp Assessment
Decomposition amp Definition
Input Output to Next Phase of
Development or Lifecycle Support Operational Need
Measures of EffectivenessSuitability
System Measures of Performance
Element Design Criteria
Spec
ialt
y En
gin
eeri
ng
Dev
elo
pm
ent
Des
ign
En
gin
eeri
ng
Dec
isio
n
Au
tho
rity
Tech Readiness
Assessment
Early TampE
Involvement
Component Level
Testing
Component
Characterization
Early Operational
Assessment
DTampE
Verification
OTampE
Validation
Source DAU Acquisition Community Connection
Early Injection of
Authoritative
systems and
mission threads
Combined DTampE
OTampE in a Joint
or SoS
Environment
Persistently Integrated
but disparately
developed capabilities
8
Sustainment Systems Acquisition (Engineering amp manufacturing development
demonstration LRIP amp production)
Pre-Systems Acquisition
IOC A B
Concept amp Tech Development
Concept
Exploration
Component
Advanced
Development
Decision
Review
System
Integration
Interim Progress Review
Production amp Deployment
LRIP
FRP Decision Review
Full-Rate Production amp Deployment
Operations
amp Support
C
The Future of Distributed Testing Test-Fix-Test
Across the Acquisition Life Cycle
Rapid Acquisition Developmental Test Operational
Test Interoperability Certification Net-Ready Key
Performance Parameters testing Joint Mission
Capability Portfolio testing
JMETC enables continuous testing across the acquisition life cycle JMETC reduces acquisition time and cost By Providing
bull Readily-available
persistent connectivity
bull Standing network
security agreements
bull Common
interoperability software
for linking sites
bull Certified test tools for
distributed testing
Enables early verification that
systems work stand alone and in a
Joint Environment
Helps find problems early in
acquisition ndash when they are less
costly to fix
Outline Distributed
Testing and JMETC
requirements in TEMP
Creates robust environment for common prototype analysis
Provides subject matter expertise to integrate distributed facilities
Test Early in Acquisition Process
Recent Distributed Test at a major test facility
bull C2 Interoperability upgrade to National Strategic asset
bull Significant software and interoperability problems found during initial Distributed Testing
bull Discovered system NOT ready for open air testing IMPACTmdashRisk Reduction Early
Identification of Problem
bullEarly Distributed Testing led to early
identification and correction of
interoperability issues
bullNo range or flying costs 9
Post-Milestone A ndash Concept Development
Joint Surface Warfare JCTD
Description
Point Mugu Test Team
demonstrated Net Enabled
Weapon Link-16 capability
using FA-18EF as launch
platform JSOW C-1 as
weapon and JSTARS as 3rd
party target source
Sample of Measures
Tactical message timing
between the controller
and the weapon
IMPACT-- Efficiency Lower Technical
Risk and Cost Savings
bull Program scheduled and executed short
multiple tests for incremental software
update evaluation
bull Resources expended on test amp analysis and
not network setup and monitoring
China Lake FA-18 JSOW
10
11
Description bull Assessment of B-2 Link-16
interoperability with AWACS
bull Connected live B-2 on ramp
at Whiteman AFB MO an
AWACS HITL at Tinker AFB
OK within a distributed C2
environment
IMPACT-- Cost Savings and Better
Product
bull Early testing led to early identification
and correction of Link 16 interoperability
issues
bull No range or flying costs
Army
Air Force
Navy
Marines
Joint
Industry
As of 14 Sep 2009
Tinker AFB AWACS
Whiteman B-2
Eglin 46TS
WPAFB SIMAF
JMETC
Langley GCIC
Army
Air Force
Navy
Marines
Joint
Industry
As of 14 Sep 2009
Tinker AFB AWACS
Whiteman B-2
Eglin 46TS
WPAFB SIMAF
JMETC
Langley GCIC
Post-Milestone C ndash Developmental Test
B-2 Spirit ICE Data Link Test
Sample of Measures
Timeliness and accuracy
of message exchanges
bull Joint Urgent Operational Need
bull Integration of BACN payload
onto multiple platforms for
solution to net-centric warfare
challenges of
bull Beyond line-of-sight comm
bull Relay bridge and range
extension for ground forces
and supporting aircraft
bull Distributed Testing included
Live-fly DT and Operational
Utility Evaluation Fall of 2010
IMPACT -- Cost Savings and
Better Product bullEfficient integration of DT and OT
bullAll required assets not available on-site
bullDistributed Testing saved ldquo$12Mrdquo
bullUrgent capability fielded
12
Responsive TampE for Rapid Fielding
Battlefield Airborne Communication Node
(BACN)
Challenges We Are Overcoming Today
Challenge
bull Site Interconnectivity
bull No common enabling
transport architecture
bull Inconsistent Tools
bull Test Participant
Collaboration
bull Analysis of Disparately
Stored Data
Overcome By
JMETC Network
TENA Middleware
JMETC ldquoBest of Breedrdquo
Process
Reuse Repository
Data Management Solutions
JMETC Joint Mission Environment Test Capability
TENA Test and Training Enabling Architecture
A TRMC Perspective
We Need a Common Vision for
Test and Training Infrastructure
Mr Derrick Hinton Principal Deputy Director Test Resource Management Center (TRMC)
Deputy Director Joint Investment Programs amp Policy (JIPP)
Office of the Undersecretary of Defense for Acquisition Technology amp Logistics (OUSD[ATampL])
Annual ITEA Test Instrumentation Workshop XIV
Test and Training Enterprise
12 May 2010
2010 2030
Individual Systems WHAT do we test Mission-Level
amp Systems of Systems Capabilities
Events WHEN do we test Continuously
Decision-Makers For WHOM do we test Decision-Makers
amp Warfighters
Agile
Streamlined
Affordable
Continuous
Goal Distributed Integrated Testing and Training
The DoD TampE process must evolve to be
Test Resource Management Center
Future TampE Process
15
Aspects of the 2030 Common
Vision
bull Test amp Training bull Real-Time TTP SOP Development to improve SoS effectiveness and
suitability (limited)
bull ldquoBlack boxrdquo and ldquowhite boxrdquo testing done in parallel
including
bull System functionality
bull System effectiveness
bull System of Systems effectiveness via mission threads in an
operationally relevant environment
bull Fewer test runsconfigurations with more meaningful
collected data
bull Real-Time Data Analysis amp Automated Reporting
OBJECTIVE Near Real-Time Test-Fix-Test 16
Some Challenges to Achieving
2030 Vision
bull Authoritative Joint Mission threads
bull Mission level measures
bull Thorough but Streamlined Information Assurance
Procedures
bull Enterprise-level Multi-Level Security solutions
bull Embedded Encryption on acquisition systems
bull RF over the wire (network)
bull Near Real Time Test-Fix-Test
bull Data Management
bull Embedded amp Unobtrusive Instrumentation
bull World wide wireless
bull Extremely efficient usage of the frequency spectrum 17
Examples Where the Vision Is Starting
to Come Together
bull Joint Integrated Air amp Missile Defense Organization
(JIAMDO) Correlation Decorrelation Interoperability Test
(CDIT) is reducing analysis cycle from 3 months to less
than a month
bull Changes to test configuration between test runs (~30 minutes)
bull Overnight AdditionSubtraction of test runs depending on issues
discovered and data analysis results
bull Real-time impact to test configuration and number of test runs
bull Emphasis on evaluators and tools for detailed data analysis
bull Air-Ground Integrated Layer Exploration (AGILE) bull AGILE provides an LVC capability for acquisition amp testing
bull Analytical Rigor provides decision quality data for decision support
bull Efficient amp collaborative use of Joint analytical resources (in
addition to service capabilities)
bull Sustains and evolves collaborative environment for future use 18
Some TRMC Investments for
Achieving Common Vision
bull Central TampE Investment Program (CTEIP)
bull Test and Training Enabling Architecture (TENA)
bull Multi-Level Security-Joint Coalition Network Environment (MLS-
JCNE)
bull InterTEC
bull Result JITC-certified instrumentation usable in all aspects of TampE
bull Common Range Integrated Instrumentation System (CRIIS)
bull TampE SampT Program
bull TENA in a Resource Constrained Environment (TRCE)
bull NST Evaluation Capability Model (NECM)
bull JMETC Program
bull DoD corporate approach for distributed testing 19
What Can You Do to Be Ready for
2030
bull Maintain Information Assurance agreements for all facility
systems
bull Streamline data collection amp analysis at your facility
bull Get comfortable with DoDAF views
bull Look across the RDTampE the community to find common
problems that can be addressed with similar solutions
bull TENA-enable your instrumentation systems and test
command amp control
bull Get involved share your requirements with your Service
representatives and in TRMC users meetings
20
JMETC Users Group
bull Purpose is to focus on technical requirements and solutions relevant to current and future Distributed Testing needs
bull Technical and Management level representatives identify core infrastructure requirements and most importantly resolve issues
bull Quarterly meetings of 250-300 JMETC customers acquisition programs test events ranges LVC sites tools and network providers
21
Last JMETC Users Group Meeting bull June 2011 bull Location Mid-East Coast bull Tracks
bull User Requirements bull Information Assurance
Security bull Data Management bull InterTEC (Current amp Planned) bull Networking
bull An established forum for the Distributed Test Community to
o Identify core infrastructure requirements and use cases
o Identify investigate amp resolve issues
o Identify opportunities to collaborate
o Discuss available solutions tools and techniques
o Share lessons learned
Summary
bull We are becoming more efficient in how we execute TampE
bull We must get the biggest and best ldquobang for the buckrdquo that we can
bull TRMC and Services continue TampE investments to enable realization of the 2030 Vision
bull Align TampE (OSD and Services) investments to mitigate test risk areas
bull The Future of TampE will include Distributed Integrated Testing and Training and will enable near real time Test-Fix-Test
22
JMETC Program Points of Contact
23
JMETC Program Manager Chip Ferguson chipfergusonosdmil 703-601-5274
JMETC Principal Deputy PM Bruce Bailey brucebaileyosdmil 703-601-5208
JMETC Lead Operations Planning Marty Arnwine martemasarnwineosdmil 703-601-5215
JMETC Senior Technical Advisor George Rumford georgerumfordosdmil
703-601-5233
JMETC Lead Systems Engineer Ryan Norman ryannormanosdmil
703-601-5277
JMETC Lead Network Engineer Arjuna ldquoAJrdquo Pathmanathan ArjunaPathmanathanosdmil
703-601-5214
JMETC Website wwwjmetcorg
Backup Slides
25
FY10 JMETC Events
Customer Event Record Event Dates
Air Force JEFX 10-1Spirit Ice (B2 Data Link Test) October ndash November 2009
Navy BAMS LVC DE October ndash December 2009
Air Force Battlefield Airborne Comm Node (BACN) JUON
(DTOT)
November 2009 to September 2010
Air Force Agile Fire 10-2 January 2010
Air Force JEFX 10-23 FebruaryApril 2010
Joint Joint Surface Warfare JCTD February to September 2010
JS J8JIAMDO Joint Sensor Integration April to September 2010
Air Force B1-B Fully Integrated Data Link Testing April 2010
JFCOM J8489 (TEST WEEK) JCAS Distributed Test June 2010
JIAMDO (Navy Lead) CorrelationDecorrelation Interoperability Test
(CDIT) Integration Events (Continuous)
July to September 2010
Army (Lead) UAS in the National Airspace July to September 2010
Air Force Agile Fire 10-3 August 2010
Joint JITC Joint Interoperability Tests JIT 10-3 amp 11-1
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Joint Strike Fighter (JSF) JIAMDOJoint Track Manager
Brigade Combat Team (BCT)
Modernization
Multi-Function Adv Data Link (MADL) Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run 25
26
JMETC FY10 Accomplishments
JMETC Accomplishments
Supported 88 distinct customer test activities
Expanded network from 38 to 57 sites
ATIN and JTDL Networks transitioned to JMETC
Upgraded JMETC support applications and utilities to TENA R6
DIACAP Tiger Team report completed and recommendations being executed
Enhanced JMETC services and capabilities provided by leveraging InterTEC Services and Industry
Reuse Repository usability improvements
FY10 Example JMETC Customers
Joint Integrated Air amp Missile Defense Organization (JIAMDO)
Broad Area Communications Node (BACN) JUON
B1-B
Broad Area Maritime Surveillance System (BAMS)
Air-Ground Integrated Layer Exploration (AGILE)
Joint Interoperability Test Command (JITC)
Selected Benefits to the DoD
Integrated DT amp OT on a Joint Urgent Operational Need for the warfighter
Maximized usage of theater assets during limited maintenance windows
Improved Joint track information sharing to ensure interoperability of systems in theater operations
Coalition exchange and examination of real-time air picture data
Identification of Air Force Initiatives ready for warfighter transition
Investigated tactical UAS deployment in the National Airspace
Employment of Net-Enabled Weapons
JCAS immediate request amp end-to-end processes ldquoas-isrdquo characterization
Determined distributed system components were not ready for full live integration testing
Executed testing to support system-of-system interoperability certification
27
Customer Event Record Event Dates
Joint JIAMDO CorrelationDecorrelation Interoperability Test (CDIT)
Integration Events (Continuous) OCONUS October 2010
Navy UAS in NAS Runs for Record October 2010
Internal InterTEC Spiral 3 Systems Acceptance Test October - November 2010
Joint JITC Joint Interoperability Tests JIT 11-12345 (Continuous) October 2010 ndash September 2011
Air Force B1-B Fully Integrated Data Link Testing November 2010
Air Force AGILE Fire Phase III JEFX 2011 December 2010 - February 2011
Joint JTRS JPO -- JTRS Ground Mobile Radio January 2011
Navy Broad Area Maritime Surveillance (BAMS) (Continuous) January ndash September 2011
Joint Joint Track Manager Capability Demonstration (Continuous) January ndash September 2011
Joint JSF Initial MampS Interoperability (Continuous) February ndash March 2011
Air Force JSTARS Interoperability Test May 2011
Joint JS J8JIAMDO Joint Sensor Integration June ndash August 2011
FY11 JMETC Events
(More to Come)
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Global Hawk GATOR
Brigade Combat Team (BCT)
Modernization F-22 FY12 Testing Planned Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run
28
JMETC Connectivity
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca (2) JITC JTDL
Redstone (7) SMDC
RTC DTCC GMAN
SED Patriot THAAD FAAD SIV
Charleston (2)IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB SIMAF
As of 16 Mar 2011
0
10
20
30
40
50
60
70
FY06 FY07 FY08 FY09 FY10 FY11
JMETC Site Expansion
Sites
Dedicated trusted connectivity on SDREN (part of the GIG)
Encrypted for Secret ndash System High
DISA-registered IP address space
Active monitoring of network performance
Capable of supporting multiple simultaneous test events Bethpage NGC BAMS
Whiteman B-2
Eglin AFB (4)AOC DTF GWEF KHILS
Hurlburt Field C2DAC
Sites in Gulf Range
Functional Sites 61
New Sites Planned 8
Connection Points to Other Networks 5
Ft Greely CRTC
Site in Alaska
MHPCC
PMRF Bldg 105
Sites in Hawaii
Camp PendletonMCTSSA
China Lake (3)AV-8B FA-18IBAR
Edwards Ridley
Corona NSWC
El Segundo NGC B-2
Sites in SoCal
Point Loma (2)RLBTSSSC-PAC 59140 SSC-PAC HAIPE
Point Mugu (2)ITEC EA-6B
West Agg Rtr
Rancho Bernardo NGC BAMS
Pax River (5)ESTEL E2CD MCLACETEF SAIL ATR JMETC SYSCONEast Agg Rtr
Aberdeen ACCN
Langley (2) C-GIIF TDLITC
JFCOM JSIC
Wallops Island (2) SCSC SSDS
DISA Sky 7
Pentagon WARCAP
Dahlgren (2) CEDLIWSL
Dam Neck CDSA
Newport News NGC VASCIC
Sites in MD DC VA
Norfolk COMOPTEVFOR
McLean MITRE NCEL
Ft Huachuca JTRS
Bethpage NGC BCN
How an Air Force Customer Sees
Distributed Connectivity
Hardware-in-the-Loop (HWIL) Availability
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca JITC
Redstone (3) DTCC GMAN SED
Charleston (2) IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB
SIMAF
Bethpage NG BAMS
Whiteman B-2
MHPCC
PMRF Bldg 105
Sites in Hawaii
All linked by JMETC
29
Challenges Explained in Detail
30
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
8
Sustainment Systems Acquisition (Engineering amp manufacturing development
demonstration LRIP amp production)
Pre-Systems Acquisition
IOC A B
Concept amp Tech Development
Concept
Exploration
Component
Advanced
Development
Decision
Review
System
Integration
Interim Progress Review
Production amp Deployment
LRIP
FRP Decision Review
Full-Rate Production amp Deployment
Operations
amp Support
C
The Future of Distributed Testing Test-Fix-Test
Across the Acquisition Life Cycle
Rapid Acquisition Developmental Test Operational
Test Interoperability Certification Net-Ready Key
Performance Parameters testing Joint Mission
Capability Portfolio testing
JMETC enables continuous testing across the acquisition life cycle JMETC reduces acquisition time and cost By Providing
bull Readily-available
persistent connectivity
bull Standing network
security agreements
bull Common
interoperability software
for linking sites
bull Certified test tools for
distributed testing
Enables early verification that
systems work stand alone and in a
Joint Environment
Helps find problems early in
acquisition ndash when they are less
costly to fix
Outline Distributed
Testing and JMETC
requirements in TEMP
Creates robust environment for common prototype analysis
Provides subject matter expertise to integrate distributed facilities
Test Early in Acquisition Process
Recent Distributed Test at a major test facility
bull C2 Interoperability upgrade to National Strategic asset
bull Significant software and interoperability problems found during initial Distributed Testing
bull Discovered system NOT ready for open air testing IMPACTmdashRisk Reduction Early
Identification of Problem
bullEarly Distributed Testing led to early
identification and correction of
interoperability issues
bullNo range or flying costs 9
Post-Milestone A ndash Concept Development
Joint Surface Warfare JCTD
Description
Point Mugu Test Team
demonstrated Net Enabled
Weapon Link-16 capability
using FA-18EF as launch
platform JSOW C-1 as
weapon and JSTARS as 3rd
party target source
Sample of Measures
Tactical message timing
between the controller
and the weapon
IMPACT-- Efficiency Lower Technical
Risk and Cost Savings
bull Program scheduled and executed short
multiple tests for incremental software
update evaluation
bull Resources expended on test amp analysis and
not network setup and monitoring
China Lake FA-18 JSOW
10
11
Description bull Assessment of B-2 Link-16
interoperability with AWACS
bull Connected live B-2 on ramp
at Whiteman AFB MO an
AWACS HITL at Tinker AFB
OK within a distributed C2
environment
IMPACT-- Cost Savings and Better
Product
bull Early testing led to early identification
and correction of Link 16 interoperability
issues
bull No range or flying costs
Army
Air Force
Navy
Marines
Joint
Industry
As of 14 Sep 2009
Tinker AFB AWACS
Whiteman B-2
Eglin 46TS
WPAFB SIMAF
JMETC
Langley GCIC
Army
Air Force
Navy
Marines
Joint
Industry
As of 14 Sep 2009
Tinker AFB AWACS
Whiteman B-2
Eglin 46TS
WPAFB SIMAF
JMETC
Langley GCIC
Post-Milestone C ndash Developmental Test
B-2 Spirit ICE Data Link Test
Sample of Measures
Timeliness and accuracy
of message exchanges
bull Joint Urgent Operational Need
bull Integration of BACN payload
onto multiple platforms for
solution to net-centric warfare
challenges of
bull Beyond line-of-sight comm
bull Relay bridge and range
extension for ground forces
and supporting aircraft
bull Distributed Testing included
Live-fly DT and Operational
Utility Evaluation Fall of 2010
IMPACT -- Cost Savings and
Better Product bullEfficient integration of DT and OT
bullAll required assets not available on-site
bullDistributed Testing saved ldquo$12Mrdquo
bullUrgent capability fielded
12
Responsive TampE for Rapid Fielding
Battlefield Airborne Communication Node
(BACN)
Challenges We Are Overcoming Today
Challenge
bull Site Interconnectivity
bull No common enabling
transport architecture
bull Inconsistent Tools
bull Test Participant
Collaboration
bull Analysis of Disparately
Stored Data
Overcome By
JMETC Network
TENA Middleware
JMETC ldquoBest of Breedrdquo
Process
Reuse Repository
Data Management Solutions
JMETC Joint Mission Environment Test Capability
TENA Test and Training Enabling Architecture
A TRMC Perspective
We Need a Common Vision for
Test and Training Infrastructure
Mr Derrick Hinton Principal Deputy Director Test Resource Management Center (TRMC)
Deputy Director Joint Investment Programs amp Policy (JIPP)
Office of the Undersecretary of Defense for Acquisition Technology amp Logistics (OUSD[ATampL])
Annual ITEA Test Instrumentation Workshop XIV
Test and Training Enterprise
12 May 2010
2010 2030
Individual Systems WHAT do we test Mission-Level
amp Systems of Systems Capabilities
Events WHEN do we test Continuously
Decision-Makers For WHOM do we test Decision-Makers
amp Warfighters
Agile
Streamlined
Affordable
Continuous
Goal Distributed Integrated Testing and Training
The DoD TampE process must evolve to be
Test Resource Management Center
Future TampE Process
15
Aspects of the 2030 Common
Vision
bull Test amp Training bull Real-Time TTP SOP Development to improve SoS effectiveness and
suitability (limited)
bull ldquoBlack boxrdquo and ldquowhite boxrdquo testing done in parallel
including
bull System functionality
bull System effectiveness
bull System of Systems effectiveness via mission threads in an
operationally relevant environment
bull Fewer test runsconfigurations with more meaningful
collected data
bull Real-Time Data Analysis amp Automated Reporting
OBJECTIVE Near Real-Time Test-Fix-Test 16
Some Challenges to Achieving
2030 Vision
bull Authoritative Joint Mission threads
bull Mission level measures
bull Thorough but Streamlined Information Assurance
Procedures
bull Enterprise-level Multi-Level Security solutions
bull Embedded Encryption on acquisition systems
bull RF over the wire (network)
bull Near Real Time Test-Fix-Test
bull Data Management
bull Embedded amp Unobtrusive Instrumentation
bull World wide wireless
bull Extremely efficient usage of the frequency spectrum 17
Examples Where the Vision Is Starting
to Come Together
bull Joint Integrated Air amp Missile Defense Organization
(JIAMDO) Correlation Decorrelation Interoperability Test
(CDIT) is reducing analysis cycle from 3 months to less
than a month
bull Changes to test configuration between test runs (~30 minutes)
bull Overnight AdditionSubtraction of test runs depending on issues
discovered and data analysis results
bull Real-time impact to test configuration and number of test runs
bull Emphasis on evaluators and tools for detailed data analysis
bull Air-Ground Integrated Layer Exploration (AGILE) bull AGILE provides an LVC capability for acquisition amp testing
bull Analytical Rigor provides decision quality data for decision support
bull Efficient amp collaborative use of Joint analytical resources (in
addition to service capabilities)
bull Sustains and evolves collaborative environment for future use 18
Some TRMC Investments for
Achieving Common Vision
bull Central TampE Investment Program (CTEIP)
bull Test and Training Enabling Architecture (TENA)
bull Multi-Level Security-Joint Coalition Network Environment (MLS-
JCNE)
bull InterTEC
bull Result JITC-certified instrumentation usable in all aspects of TampE
bull Common Range Integrated Instrumentation System (CRIIS)
bull TampE SampT Program
bull TENA in a Resource Constrained Environment (TRCE)
bull NST Evaluation Capability Model (NECM)
bull JMETC Program
bull DoD corporate approach for distributed testing 19
What Can You Do to Be Ready for
2030
bull Maintain Information Assurance agreements for all facility
systems
bull Streamline data collection amp analysis at your facility
bull Get comfortable with DoDAF views
bull Look across the RDTampE the community to find common
problems that can be addressed with similar solutions
bull TENA-enable your instrumentation systems and test
command amp control
bull Get involved share your requirements with your Service
representatives and in TRMC users meetings
20
JMETC Users Group
bull Purpose is to focus on technical requirements and solutions relevant to current and future Distributed Testing needs
bull Technical and Management level representatives identify core infrastructure requirements and most importantly resolve issues
bull Quarterly meetings of 250-300 JMETC customers acquisition programs test events ranges LVC sites tools and network providers
21
Last JMETC Users Group Meeting bull June 2011 bull Location Mid-East Coast bull Tracks
bull User Requirements bull Information Assurance
Security bull Data Management bull InterTEC (Current amp Planned) bull Networking
bull An established forum for the Distributed Test Community to
o Identify core infrastructure requirements and use cases
o Identify investigate amp resolve issues
o Identify opportunities to collaborate
o Discuss available solutions tools and techniques
o Share lessons learned
Summary
bull We are becoming more efficient in how we execute TampE
bull We must get the biggest and best ldquobang for the buckrdquo that we can
bull TRMC and Services continue TampE investments to enable realization of the 2030 Vision
bull Align TampE (OSD and Services) investments to mitigate test risk areas
bull The Future of TampE will include Distributed Integrated Testing and Training and will enable near real time Test-Fix-Test
22
JMETC Program Points of Contact
23
JMETC Program Manager Chip Ferguson chipfergusonosdmil 703-601-5274
JMETC Principal Deputy PM Bruce Bailey brucebaileyosdmil 703-601-5208
JMETC Lead Operations Planning Marty Arnwine martemasarnwineosdmil 703-601-5215
JMETC Senior Technical Advisor George Rumford georgerumfordosdmil
703-601-5233
JMETC Lead Systems Engineer Ryan Norman ryannormanosdmil
703-601-5277
JMETC Lead Network Engineer Arjuna ldquoAJrdquo Pathmanathan ArjunaPathmanathanosdmil
703-601-5214
JMETC Website wwwjmetcorg
Backup Slides
25
FY10 JMETC Events
Customer Event Record Event Dates
Air Force JEFX 10-1Spirit Ice (B2 Data Link Test) October ndash November 2009
Navy BAMS LVC DE October ndash December 2009
Air Force Battlefield Airborne Comm Node (BACN) JUON
(DTOT)
November 2009 to September 2010
Air Force Agile Fire 10-2 January 2010
Air Force JEFX 10-23 FebruaryApril 2010
Joint Joint Surface Warfare JCTD February to September 2010
JS J8JIAMDO Joint Sensor Integration April to September 2010
Air Force B1-B Fully Integrated Data Link Testing April 2010
JFCOM J8489 (TEST WEEK) JCAS Distributed Test June 2010
JIAMDO (Navy Lead) CorrelationDecorrelation Interoperability Test
(CDIT) Integration Events (Continuous)
July to September 2010
Army (Lead) UAS in the National Airspace July to September 2010
Air Force Agile Fire 10-3 August 2010
Joint JITC Joint Interoperability Tests JIT 10-3 amp 11-1
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Joint Strike Fighter (JSF) JIAMDOJoint Track Manager
Brigade Combat Team (BCT)
Modernization
Multi-Function Adv Data Link (MADL) Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run 25
26
JMETC FY10 Accomplishments
JMETC Accomplishments
Supported 88 distinct customer test activities
Expanded network from 38 to 57 sites
ATIN and JTDL Networks transitioned to JMETC
Upgraded JMETC support applications and utilities to TENA R6
DIACAP Tiger Team report completed and recommendations being executed
Enhanced JMETC services and capabilities provided by leveraging InterTEC Services and Industry
Reuse Repository usability improvements
FY10 Example JMETC Customers
Joint Integrated Air amp Missile Defense Organization (JIAMDO)
Broad Area Communications Node (BACN) JUON
B1-B
Broad Area Maritime Surveillance System (BAMS)
Air-Ground Integrated Layer Exploration (AGILE)
Joint Interoperability Test Command (JITC)
Selected Benefits to the DoD
Integrated DT amp OT on a Joint Urgent Operational Need for the warfighter
Maximized usage of theater assets during limited maintenance windows
Improved Joint track information sharing to ensure interoperability of systems in theater operations
Coalition exchange and examination of real-time air picture data
Identification of Air Force Initiatives ready for warfighter transition
Investigated tactical UAS deployment in the National Airspace
Employment of Net-Enabled Weapons
JCAS immediate request amp end-to-end processes ldquoas-isrdquo characterization
Determined distributed system components were not ready for full live integration testing
Executed testing to support system-of-system interoperability certification
27
Customer Event Record Event Dates
Joint JIAMDO CorrelationDecorrelation Interoperability Test (CDIT)
Integration Events (Continuous) OCONUS October 2010
Navy UAS in NAS Runs for Record October 2010
Internal InterTEC Spiral 3 Systems Acceptance Test October - November 2010
Joint JITC Joint Interoperability Tests JIT 11-12345 (Continuous) October 2010 ndash September 2011
Air Force B1-B Fully Integrated Data Link Testing November 2010
Air Force AGILE Fire Phase III JEFX 2011 December 2010 - February 2011
Joint JTRS JPO -- JTRS Ground Mobile Radio January 2011
Navy Broad Area Maritime Surveillance (BAMS) (Continuous) January ndash September 2011
Joint Joint Track Manager Capability Demonstration (Continuous) January ndash September 2011
Joint JSF Initial MampS Interoperability (Continuous) February ndash March 2011
Air Force JSTARS Interoperability Test May 2011
Joint JS J8JIAMDO Joint Sensor Integration June ndash August 2011
FY11 JMETC Events
(More to Come)
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Global Hawk GATOR
Brigade Combat Team (BCT)
Modernization F-22 FY12 Testing Planned Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run
28
JMETC Connectivity
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca (2) JITC JTDL
Redstone (7) SMDC
RTC DTCC GMAN
SED Patriot THAAD FAAD SIV
Charleston (2)IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB SIMAF
As of 16 Mar 2011
0
10
20
30
40
50
60
70
FY06 FY07 FY08 FY09 FY10 FY11
JMETC Site Expansion
Sites
Dedicated trusted connectivity on SDREN (part of the GIG)
Encrypted for Secret ndash System High
DISA-registered IP address space
Active monitoring of network performance
Capable of supporting multiple simultaneous test events Bethpage NGC BAMS
Whiteman B-2
Eglin AFB (4)AOC DTF GWEF KHILS
Hurlburt Field C2DAC
Sites in Gulf Range
Functional Sites 61
New Sites Planned 8
Connection Points to Other Networks 5
Ft Greely CRTC
Site in Alaska
MHPCC
PMRF Bldg 105
Sites in Hawaii
Camp PendletonMCTSSA
China Lake (3)AV-8B FA-18IBAR
Edwards Ridley
Corona NSWC
El Segundo NGC B-2
Sites in SoCal
Point Loma (2)RLBTSSSC-PAC 59140 SSC-PAC HAIPE
Point Mugu (2)ITEC EA-6B
West Agg Rtr
Rancho Bernardo NGC BAMS
Pax River (5)ESTEL E2CD MCLACETEF SAIL ATR JMETC SYSCONEast Agg Rtr
Aberdeen ACCN
Langley (2) C-GIIF TDLITC
JFCOM JSIC
Wallops Island (2) SCSC SSDS
DISA Sky 7
Pentagon WARCAP
Dahlgren (2) CEDLIWSL
Dam Neck CDSA
Newport News NGC VASCIC
Sites in MD DC VA
Norfolk COMOPTEVFOR
McLean MITRE NCEL
Ft Huachuca JTRS
Bethpage NGC BCN
How an Air Force Customer Sees
Distributed Connectivity
Hardware-in-the-Loop (HWIL) Availability
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca JITC
Redstone (3) DTCC GMAN SED
Charleston (2) IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB
SIMAF
Bethpage NG BAMS
Whiteman B-2
MHPCC
PMRF Bldg 105
Sites in Hawaii
All linked by JMETC
29
Challenges Explained in Detail
30
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
Test Early in Acquisition Process
Recent Distributed Test at a major test facility
bull C2 Interoperability upgrade to National Strategic asset
bull Significant software and interoperability problems found during initial Distributed Testing
bull Discovered system NOT ready for open air testing IMPACTmdashRisk Reduction Early
Identification of Problem
bullEarly Distributed Testing led to early
identification and correction of
interoperability issues
bullNo range or flying costs 9
Post-Milestone A ndash Concept Development
Joint Surface Warfare JCTD
Description
Point Mugu Test Team
demonstrated Net Enabled
Weapon Link-16 capability
using FA-18EF as launch
platform JSOW C-1 as
weapon and JSTARS as 3rd
party target source
Sample of Measures
Tactical message timing
between the controller
and the weapon
IMPACT-- Efficiency Lower Technical
Risk and Cost Savings
bull Program scheduled and executed short
multiple tests for incremental software
update evaluation
bull Resources expended on test amp analysis and
not network setup and monitoring
China Lake FA-18 JSOW
10
11
Description bull Assessment of B-2 Link-16
interoperability with AWACS
bull Connected live B-2 on ramp
at Whiteman AFB MO an
AWACS HITL at Tinker AFB
OK within a distributed C2
environment
IMPACT-- Cost Savings and Better
Product
bull Early testing led to early identification
and correction of Link 16 interoperability
issues
bull No range or flying costs
Army
Air Force
Navy
Marines
Joint
Industry
As of 14 Sep 2009
Tinker AFB AWACS
Whiteman B-2
Eglin 46TS
WPAFB SIMAF
JMETC
Langley GCIC
Army
Air Force
Navy
Marines
Joint
Industry
As of 14 Sep 2009
Tinker AFB AWACS
Whiteman B-2
Eglin 46TS
WPAFB SIMAF
JMETC
Langley GCIC
Post-Milestone C ndash Developmental Test
B-2 Spirit ICE Data Link Test
Sample of Measures
Timeliness and accuracy
of message exchanges
bull Joint Urgent Operational Need
bull Integration of BACN payload
onto multiple platforms for
solution to net-centric warfare
challenges of
bull Beyond line-of-sight comm
bull Relay bridge and range
extension for ground forces
and supporting aircraft
bull Distributed Testing included
Live-fly DT and Operational
Utility Evaluation Fall of 2010
IMPACT -- Cost Savings and
Better Product bullEfficient integration of DT and OT
bullAll required assets not available on-site
bullDistributed Testing saved ldquo$12Mrdquo
bullUrgent capability fielded
12
Responsive TampE for Rapid Fielding
Battlefield Airborne Communication Node
(BACN)
Challenges We Are Overcoming Today
Challenge
bull Site Interconnectivity
bull No common enabling
transport architecture
bull Inconsistent Tools
bull Test Participant
Collaboration
bull Analysis of Disparately
Stored Data
Overcome By
JMETC Network
TENA Middleware
JMETC ldquoBest of Breedrdquo
Process
Reuse Repository
Data Management Solutions
JMETC Joint Mission Environment Test Capability
TENA Test and Training Enabling Architecture
A TRMC Perspective
We Need a Common Vision for
Test and Training Infrastructure
Mr Derrick Hinton Principal Deputy Director Test Resource Management Center (TRMC)
Deputy Director Joint Investment Programs amp Policy (JIPP)
Office of the Undersecretary of Defense for Acquisition Technology amp Logistics (OUSD[ATampL])
Annual ITEA Test Instrumentation Workshop XIV
Test and Training Enterprise
12 May 2010
2010 2030
Individual Systems WHAT do we test Mission-Level
amp Systems of Systems Capabilities
Events WHEN do we test Continuously
Decision-Makers For WHOM do we test Decision-Makers
amp Warfighters
Agile
Streamlined
Affordable
Continuous
Goal Distributed Integrated Testing and Training
The DoD TampE process must evolve to be
Test Resource Management Center
Future TampE Process
15
Aspects of the 2030 Common
Vision
bull Test amp Training bull Real-Time TTP SOP Development to improve SoS effectiveness and
suitability (limited)
bull ldquoBlack boxrdquo and ldquowhite boxrdquo testing done in parallel
including
bull System functionality
bull System effectiveness
bull System of Systems effectiveness via mission threads in an
operationally relevant environment
bull Fewer test runsconfigurations with more meaningful
collected data
bull Real-Time Data Analysis amp Automated Reporting
OBJECTIVE Near Real-Time Test-Fix-Test 16
Some Challenges to Achieving
2030 Vision
bull Authoritative Joint Mission threads
bull Mission level measures
bull Thorough but Streamlined Information Assurance
Procedures
bull Enterprise-level Multi-Level Security solutions
bull Embedded Encryption on acquisition systems
bull RF over the wire (network)
bull Near Real Time Test-Fix-Test
bull Data Management
bull Embedded amp Unobtrusive Instrumentation
bull World wide wireless
bull Extremely efficient usage of the frequency spectrum 17
Examples Where the Vision Is Starting
to Come Together
bull Joint Integrated Air amp Missile Defense Organization
(JIAMDO) Correlation Decorrelation Interoperability Test
(CDIT) is reducing analysis cycle from 3 months to less
than a month
bull Changes to test configuration between test runs (~30 minutes)
bull Overnight AdditionSubtraction of test runs depending on issues
discovered and data analysis results
bull Real-time impact to test configuration and number of test runs
bull Emphasis on evaluators and tools for detailed data analysis
bull Air-Ground Integrated Layer Exploration (AGILE) bull AGILE provides an LVC capability for acquisition amp testing
bull Analytical Rigor provides decision quality data for decision support
bull Efficient amp collaborative use of Joint analytical resources (in
addition to service capabilities)
bull Sustains and evolves collaborative environment for future use 18
Some TRMC Investments for
Achieving Common Vision
bull Central TampE Investment Program (CTEIP)
bull Test and Training Enabling Architecture (TENA)
bull Multi-Level Security-Joint Coalition Network Environment (MLS-
JCNE)
bull InterTEC
bull Result JITC-certified instrumentation usable in all aspects of TampE
bull Common Range Integrated Instrumentation System (CRIIS)
bull TampE SampT Program
bull TENA in a Resource Constrained Environment (TRCE)
bull NST Evaluation Capability Model (NECM)
bull JMETC Program
bull DoD corporate approach for distributed testing 19
What Can You Do to Be Ready for
2030
bull Maintain Information Assurance agreements for all facility
systems
bull Streamline data collection amp analysis at your facility
bull Get comfortable with DoDAF views
bull Look across the RDTampE the community to find common
problems that can be addressed with similar solutions
bull TENA-enable your instrumentation systems and test
command amp control
bull Get involved share your requirements with your Service
representatives and in TRMC users meetings
20
JMETC Users Group
bull Purpose is to focus on technical requirements and solutions relevant to current and future Distributed Testing needs
bull Technical and Management level representatives identify core infrastructure requirements and most importantly resolve issues
bull Quarterly meetings of 250-300 JMETC customers acquisition programs test events ranges LVC sites tools and network providers
21
Last JMETC Users Group Meeting bull June 2011 bull Location Mid-East Coast bull Tracks
bull User Requirements bull Information Assurance
Security bull Data Management bull InterTEC (Current amp Planned) bull Networking
bull An established forum for the Distributed Test Community to
o Identify core infrastructure requirements and use cases
o Identify investigate amp resolve issues
o Identify opportunities to collaborate
o Discuss available solutions tools and techniques
o Share lessons learned
Summary
bull We are becoming more efficient in how we execute TampE
bull We must get the biggest and best ldquobang for the buckrdquo that we can
bull TRMC and Services continue TampE investments to enable realization of the 2030 Vision
bull Align TampE (OSD and Services) investments to mitigate test risk areas
bull The Future of TampE will include Distributed Integrated Testing and Training and will enable near real time Test-Fix-Test
22
JMETC Program Points of Contact
23
JMETC Program Manager Chip Ferguson chipfergusonosdmil 703-601-5274
JMETC Principal Deputy PM Bruce Bailey brucebaileyosdmil 703-601-5208
JMETC Lead Operations Planning Marty Arnwine martemasarnwineosdmil 703-601-5215
JMETC Senior Technical Advisor George Rumford georgerumfordosdmil
703-601-5233
JMETC Lead Systems Engineer Ryan Norman ryannormanosdmil
703-601-5277
JMETC Lead Network Engineer Arjuna ldquoAJrdquo Pathmanathan ArjunaPathmanathanosdmil
703-601-5214
JMETC Website wwwjmetcorg
Backup Slides
25
FY10 JMETC Events
Customer Event Record Event Dates
Air Force JEFX 10-1Spirit Ice (B2 Data Link Test) October ndash November 2009
Navy BAMS LVC DE October ndash December 2009
Air Force Battlefield Airborne Comm Node (BACN) JUON
(DTOT)
November 2009 to September 2010
Air Force Agile Fire 10-2 January 2010
Air Force JEFX 10-23 FebruaryApril 2010
Joint Joint Surface Warfare JCTD February to September 2010
JS J8JIAMDO Joint Sensor Integration April to September 2010
Air Force B1-B Fully Integrated Data Link Testing April 2010
JFCOM J8489 (TEST WEEK) JCAS Distributed Test June 2010
JIAMDO (Navy Lead) CorrelationDecorrelation Interoperability Test
(CDIT) Integration Events (Continuous)
July to September 2010
Army (Lead) UAS in the National Airspace July to September 2010
Air Force Agile Fire 10-3 August 2010
Joint JITC Joint Interoperability Tests JIT 10-3 amp 11-1
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Joint Strike Fighter (JSF) JIAMDOJoint Track Manager
Brigade Combat Team (BCT)
Modernization
Multi-Function Adv Data Link (MADL) Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run 25
26
JMETC FY10 Accomplishments
JMETC Accomplishments
Supported 88 distinct customer test activities
Expanded network from 38 to 57 sites
ATIN and JTDL Networks transitioned to JMETC
Upgraded JMETC support applications and utilities to TENA R6
DIACAP Tiger Team report completed and recommendations being executed
Enhanced JMETC services and capabilities provided by leveraging InterTEC Services and Industry
Reuse Repository usability improvements
FY10 Example JMETC Customers
Joint Integrated Air amp Missile Defense Organization (JIAMDO)
Broad Area Communications Node (BACN) JUON
B1-B
Broad Area Maritime Surveillance System (BAMS)
Air-Ground Integrated Layer Exploration (AGILE)
Joint Interoperability Test Command (JITC)
Selected Benefits to the DoD
Integrated DT amp OT on a Joint Urgent Operational Need for the warfighter
Maximized usage of theater assets during limited maintenance windows
Improved Joint track information sharing to ensure interoperability of systems in theater operations
Coalition exchange and examination of real-time air picture data
Identification of Air Force Initiatives ready for warfighter transition
Investigated tactical UAS deployment in the National Airspace
Employment of Net-Enabled Weapons
JCAS immediate request amp end-to-end processes ldquoas-isrdquo characterization
Determined distributed system components were not ready for full live integration testing
Executed testing to support system-of-system interoperability certification
27
Customer Event Record Event Dates
Joint JIAMDO CorrelationDecorrelation Interoperability Test (CDIT)
Integration Events (Continuous) OCONUS October 2010
Navy UAS in NAS Runs for Record October 2010
Internal InterTEC Spiral 3 Systems Acceptance Test October - November 2010
Joint JITC Joint Interoperability Tests JIT 11-12345 (Continuous) October 2010 ndash September 2011
Air Force B1-B Fully Integrated Data Link Testing November 2010
Air Force AGILE Fire Phase III JEFX 2011 December 2010 - February 2011
Joint JTRS JPO -- JTRS Ground Mobile Radio January 2011
Navy Broad Area Maritime Surveillance (BAMS) (Continuous) January ndash September 2011
Joint Joint Track Manager Capability Demonstration (Continuous) January ndash September 2011
Joint JSF Initial MampS Interoperability (Continuous) February ndash March 2011
Air Force JSTARS Interoperability Test May 2011
Joint JS J8JIAMDO Joint Sensor Integration June ndash August 2011
FY11 JMETC Events
(More to Come)
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Global Hawk GATOR
Brigade Combat Team (BCT)
Modernization F-22 FY12 Testing Planned Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run
28
JMETC Connectivity
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca (2) JITC JTDL
Redstone (7) SMDC
RTC DTCC GMAN
SED Patriot THAAD FAAD SIV
Charleston (2)IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB SIMAF
As of 16 Mar 2011
0
10
20
30
40
50
60
70
FY06 FY07 FY08 FY09 FY10 FY11
JMETC Site Expansion
Sites
Dedicated trusted connectivity on SDREN (part of the GIG)
Encrypted for Secret ndash System High
DISA-registered IP address space
Active monitoring of network performance
Capable of supporting multiple simultaneous test events Bethpage NGC BAMS
Whiteman B-2
Eglin AFB (4)AOC DTF GWEF KHILS
Hurlburt Field C2DAC
Sites in Gulf Range
Functional Sites 61
New Sites Planned 8
Connection Points to Other Networks 5
Ft Greely CRTC
Site in Alaska
MHPCC
PMRF Bldg 105
Sites in Hawaii
Camp PendletonMCTSSA
China Lake (3)AV-8B FA-18IBAR
Edwards Ridley
Corona NSWC
El Segundo NGC B-2
Sites in SoCal
Point Loma (2)RLBTSSSC-PAC 59140 SSC-PAC HAIPE
Point Mugu (2)ITEC EA-6B
West Agg Rtr
Rancho Bernardo NGC BAMS
Pax River (5)ESTEL E2CD MCLACETEF SAIL ATR JMETC SYSCONEast Agg Rtr
Aberdeen ACCN
Langley (2) C-GIIF TDLITC
JFCOM JSIC
Wallops Island (2) SCSC SSDS
DISA Sky 7
Pentagon WARCAP
Dahlgren (2) CEDLIWSL
Dam Neck CDSA
Newport News NGC VASCIC
Sites in MD DC VA
Norfolk COMOPTEVFOR
McLean MITRE NCEL
Ft Huachuca JTRS
Bethpage NGC BCN
How an Air Force Customer Sees
Distributed Connectivity
Hardware-in-the-Loop (HWIL) Availability
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca JITC
Redstone (3) DTCC GMAN SED
Charleston (2) IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB
SIMAF
Bethpage NG BAMS
Whiteman B-2
MHPCC
PMRF Bldg 105
Sites in Hawaii
All linked by JMETC
29
Challenges Explained in Detail
30
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
Post-Milestone A ndash Concept Development
Joint Surface Warfare JCTD
Description
Point Mugu Test Team
demonstrated Net Enabled
Weapon Link-16 capability
using FA-18EF as launch
platform JSOW C-1 as
weapon and JSTARS as 3rd
party target source
Sample of Measures
Tactical message timing
between the controller
and the weapon
IMPACT-- Efficiency Lower Technical
Risk and Cost Savings
bull Program scheduled and executed short
multiple tests for incremental software
update evaluation
bull Resources expended on test amp analysis and
not network setup and monitoring
China Lake FA-18 JSOW
10
11
Description bull Assessment of B-2 Link-16
interoperability with AWACS
bull Connected live B-2 on ramp
at Whiteman AFB MO an
AWACS HITL at Tinker AFB
OK within a distributed C2
environment
IMPACT-- Cost Savings and Better
Product
bull Early testing led to early identification
and correction of Link 16 interoperability
issues
bull No range or flying costs
Army
Air Force
Navy
Marines
Joint
Industry
As of 14 Sep 2009
Tinker AFB AWACS
Whiteman B-2
Eglin 46TS
WPAFB SIMAF
JMETC
Langley GCIC
Army
Air Force
Navy
Marines
Joint
Industry
As of 14 Sep 2009
Tinker AFB AWACS
Whiteman B-2
Eglin 46TS
WPAFB SIMAF
JMETC
Langley GCIC
Post-Milestone C ndash Developmental Test
B-2 Spirit ICE Data Link Test
Sample of Measures
Timeliness and accuracy
of message exchanges
bull Joint Urgent Operational Need
bull Integration of BACN payload
onto multiple platforms for
solution to net-centric warfare
challenges of
bull Beyond line-of-sight comm
bull Relay bridge and range
extension for ground forces
and supporting aircraft
bull Distributed Testing included
Live-fly DT and Operational
Utility Evaluation Fall of 2010
IMPACT -- Cost Savings and
Better Product bullEfficient integration of DT and OT
bullAll required assets not available on-site
bullDistributed Testing saved ldquo$12Mrdquo
bullUrgent capability fielded
12
Responsive TampE for Rapid Fielding
Battlefield Airborne Communication Node
(BACN)
Challenges We Are Overcoming Today
Challenge
bull Site Interconnectivity
bull No common enabling
transport architecture
bull Inconsistent Tools
bull Test Participant
Collaboration
bull Analysis of Disparately
Stored Data
Overcome By
JMETC Network
TENA Middleware
JMETC ldquoBest of Breedrdquo
Process
Reuse Repository
Data Management Solutions
JMETC Joint Mission Environment Test Capability
TENA Test and Training Enabling Architecture
A TRMC Perspective
We Need a Common Vision for
Test and Training Infrastructure
Mr Derrick Hinton Principal Deputy Director Test Resource Management Center (TRMC)
Deputy Director Joint Investment Programs amp Policy (JIPP)
Office of the Undersecretary of Defense for Acquisition Technology amp Logistics (OUSD[ATampL])
Annual ITEA Test Instrumentation Workshop XIV
Test and Training Enterprise
12 May 2010
2010 2030
Individual Systems WHAT do we test Mission-Level
amp Systems of Systems Capabilities
Events WHEN do we test Continuously
Decision-Makers For WHOM do we test Decision-Makers
amp Warfighters
Agile
Streamlined
Affordable
Continuous
Goal Distributed Integrated Testing and Training
The DoD TampE process must evolve to be
Test Resource Management Center
Future TampE Process
15
Aspects of the 2030 Common
Vision
bull Test amp Training bull Real-Time TTP SOP Development to improve SoS effectiveness and
suitability (limited)
bull ldquoBlack boxrdquo and ldquowhite boxrdquo testing done in parallel
including
bull System functionality
bull System effectiveness
bull System of Systems effectiveness via mission threads in an
operationally relevant environment
bull Fewer test runsconfigurations with more meaningful
collected data
bull Real-Time Data Analysis amp Automated Reporting
OBJECTIVE Near Real-Time Test-Fix-Test 16
Some Challenges to Achieving
2030 Vision
bull Authoritative Joint Mission threads
bull Mission level measures
bull Thorough but Streamlined Information Assurance
Procedures
bull Enterprise-level Multi-Level Security solutions
bull Embedded Encryption on acquisition systems
bull RF over the wire (network)
bull Near Real Time Test-Fix-Test
bull Data Management
bull Embedded amp Unobtrusive Instrumentation
bull World wide wireless
bull Extremely efficient usage of the frequency spectrum 17
Examples Where the Vision Is Starting
to Come Together
bull Joint Integrated Air amp Missile Defense Organization
(JIAMDO) Correlation Decorrelation Interoperability Test
(CDIT) is reducing analysis cycle from 3 months to less
than a month
bull Changes to test configuration between test runs (~30 minutes)
bull Overnight AdditionSubtraction of test runs depending on issues
discovered and data analysis results
bull Real-time impact to test configuration and number of test runs
bull Emphasis on evaluators and tools for detailed data analysis
bull Air-Ground Integrated Layer Exploration (AGILE) bull AGILE provides an LVC capability for acquisition amp testing
bull Analytical Rigor provides decision quality data for decision support
bull Efficient amp collaborative use of Joint analytical resources (in
addition to service capabilities)
bull Sustains and evolves collaborative environment for future use 18
Some TRMC Investments for
Achieving Common Vision
bull Central TampE Investment Program (CTEIP)
bull Test and Training Enabling Architecture (TENA)
bull Multi-Level Security-Joint Coalition Network Environment (MLS-
JCNE)
bull InterTEC
bull Result JITC-certified instrumentation usable in all aspects of TampE
bull Common Range Integrated Instrumentation System (CRIIS)
bull TampE SampT Program
bull TENA in a Resource Constrained Environment (TRCE)
bull NST Evaluation Capability Model (NECM)
bull JMETC Program
bull DoD corporate approach for distributed testing 19
What Can You Do to Be Ready for
2030
bull Maintain Information Assurance agreements for all facility
systems
bull Streamline data collection amp analysis at your facility
bull Get comfortable with DoDAF views
bull Look across the RDTampE the community to find common
problems that can be addressed with similar solutions
bull TENA-enable your instrumentation systems and test
command amp control
bull Get involved share your requirements with your Service
representatives and in TRMC users meetings
20
JMETC Users Group
bull Purpose is to focus on technical requirements and solutions relevant to current and future Distributed Testing needs
bull Technical and Management level representatives identify core infrastructure requirements and most importantly resolve issues
bull Quarterly meetings of 250-300 JMETC customers acquisition programs test events ranges LVC sites tools and network providers
21
Last JMETC Users Group Meeting bull June 2011 bull Location Mid-East Coast bull Tracks
bull User Requirements bull Information Assurance
Security bull Data Management bull InterTEC (Current amp Planned) bull Networking
bull An established forum for the Distributed Test Community to
o Identify core infrastructure requirements and use cases
o Identify investigate amp resolve issues
o Identify opportunities to collaborate
o Discuss available solutions tools and techniques
o Share lessons learned
Summary
bull We are becoming more efficient in how we execute TampE
bull We must get the biggest and best ldquobang for the buckrdquo that we can
bull TRMC and Services continue TampE investments to enable realization of the 2030 Vision
bull Align TampE (OSD and Services) investments to mitigate test risk areas
bull The Future of TampE will include Distributed Integrated Testing and Training and will enable near real time Test-Fix-Test
22
JMETC Program Points of Contact
23
JMETC Program Manager Chip Ferguson chipfergusonosdmil 703-601-5274
JMETC Principal Deputy PM Bruce Bailey brucebaileyosdmil 703-601-5208
JMETC Lead Operations Planning Marty Arnwine martemasarnwineosdmil 703-601-5215
JMETC Senior Technical Advisor George Rumford georgerumfordosdmil
703-601-5233
JMETC Lead Systems Engineer Ryan Norman ryannormanosdmil
703-601-5277
JMETC Lead Network Engineer Arjuna ldquoAJrdquo Pathmanathan ArjunaPathmanathanosdmil
703-601-5214
JMETC Website wwwjmetcorg
Backup Slides
25
FY10 JMETC Events
Customer Event Record Event Dates
Air Force JEFX 10-1Spirit Ice (B2 Data Link Test) October ndash November 2009
Navy BAMS LVC DE October ndash December 2009
Air Force Battlefield Airborne Comm Node (BACN) JUON
(DTOT)
November 2009 to September 2010
Air Force Agile Fire 10-2 January 2010
Air Force JEFX 10-23 FebruaryApril 2010
Joint Joint Surface Warfare JCTD February to September 2010
JS J8JIAMDO Joint Sensor Integration April to September 2010
Air Force B1-B Fully Integrated Data Link Testing April 2010
JFCOM J8489 (TEST WEEK) JCAS Distributed Test June 2010
JIAMDO (Navy Lead) CorrelationDecorrelation Interoperability Test
(CDIT) Integration Events (Continuous)
July to September 2010
Army (Lead) UAS in the National Airspace July to September 2010
Air Force Agile Fire 10-3 August 2010
Joint JITC Joint Interoperability Tests JIT 10-3 amp 11-1
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Joint Strike Fighter (JSF) JIAMDOJoint Track Manager
Brigade Combat Team (BCT)
Modernization
Multi-Function Adv Data Link (MADL) Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run 25
26
JMETC FY10 Accomplishments
JMETC Accomplishments
Supported 88 distinct customer test activities
Expanded network from 38 to 57 sites
ATIN and JTDL Networks transitioned to JMETC
Upgraded JMETC support applications and utilities to TENA R6
DIACAP Tiger Team report completed and recommendations being executed
Enhanced JMETC services and capabilities provided by leveraging InterTEC Services and Industry
Reuse Repository usability improvements
FY10 Example JMETC Customers
Joint Integrated Air amp Missile Defense Organization (JIAMDO)
Broad Area Communications Node (BACN) JUON
B1-B
Broad Area Maritime Surveillance System (BAMS)
Air-Ground Integrated Layer Exploration (AGILE)
Joint Interoperability Test Command (JITC)
Selected Benefits to the DoD
Integrated DT amp OT on a Joint Urgent Operational Need for the warfighter
Maximized usage of theater assets during limited maintenance windows
Improved Joint track information sharing to ensure interoperability of systems in theater operations
Coalition exchange and examination of real-time air picture data
Identification of Air Force Initiatives ready for warfighter transition
Investigated tactical UAS deployment in the National Airspace
Employment of Net-Enabled Weapons
JCAS immediate request amp end-to-end processes ldquoas-isrdquo characterization
Determined distributed system components were not ready for full live integration testing
Executed testing to support system-of-system interoperability certification
27
Customer Event Record Event Dates
Joint JIAMDO CorrelationDecorrelation Interoperability Test (CDIT)
Integration Events (Continuous) OCONUS October 2010
Navy UAS in NAS Runs for Record October 2010
Internal InterTEC Spiral 3 Systems Acceptance Test October - November 2010
Joint JITC Joint Interoperability Tests JIT 11-12345 (Continuous) October 2010 ndash September 2011
Air Force B1-B Fully Integrated Data Link Testing November 2010
Air Force AGILE Fire Phase III JEFX 2011 December 2010 - February 2011
Joint JTRS JPO -- JTRS Ground Mobile Radio January 2011
Navy Broad Area Maritime Surveillance (BAMS) (Continuous) January ndash September 2011
Joint Joint Track Manager Capability Demonstration (Continuous) January ndash September 2011
Joint JSF Initial MampS Interoperability (Continuous) February ndash March 2011
Air Force JSTARS Interoperability Test May 2011
Joint JS J8JIAMDO Joint Sensor Integration June ndash August 2011
FY11 JMETC Events
(More to Come)
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Global Hawk GATOR
Brigade Combat Team (BCT)
Modernization F-22 FY12 Testing Planned Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run
28
JMETC Connectivity
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca (2) JITC JTDL
Redstone (7) SMDC
RTC DTCC GMAN
SED Patriot THAAD FAAD SIV
Charleston (2)IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB SIMAF
As of 16 Mar 2011
0
10
20
30
40
50
60
70
FY06 FY07 FY08 FY09 FY10 FY11
JMETC Site Expansion
Sites
Dedicated trusted connectivity on SDREN (part of the GIG)
Encrypted for Secret ndash System High
DISA-registered IP address space
Active monitoring of network performance
Capable of supporting multiple simultaneous test events Bethpage NGC BAMS
Whiteman B-2
Eglin AFB (4)AOC DTF GWEF KHILS
Hurlburt Field C2DAC
Sites in Gulf Range
Functional Sites 61
New Sites Planned 8
Connection Points to Other Networks 5
Ft Greely CRTC
Site in Alaska
MHPCC
PMRF Bldg 105
Sites in Hawaii
Camp PendletonMCTSSA
China Lake (3)AV-8B FA-18IBAR
Edwards Ridley
Corona NSWC
El Segundo NGC B-2
Sites in SoCal
Point Loma (2)RLBTSSSC-PAC 59140 SSC-PAC HAIPE
Point Mugu (2)ITEC EA-6B
West Agg Rtr
Rancho Bernardo NGC BAMS
Pax River (5)ESTEL E2CD MCLACETEF SAIL ATR JMETC SYSCONEast Agg Rtr
Aberdeen ACCN
Langley (2) C-GIIF TDLITC
JFCOM JSIC
Wallops Island (2) SCSC SSDS
DISA Sky 7
Pentagon WARCAP
Dahlgren (2) CEDLIWSL
Dam Neck CDSA
Newport News NGC VASCIC
Sites in MD DC VA
Norfolk COMOPTEVFOR
McLean MITRE NCEL
Ft Huachuca JTRS
Bethpage NGC BCN
How an Air Force Customer Sees
Distributed Connectivity
Hardware-in-the-Loop (HWIL) Availability
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca JITC
Redstone (3) DTCC GMAN SED
Charleston (2) IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB
SIMAF
Bethpage NG BAMS
Whiteman B-2
MHPCC
PMRF Bldg 105
Sites in Hawaii
All linked by JMETC
29
Challenges Explained in Detail
30
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
11
Description bull Assessment of B-2 Link-16
interoperability with AWACS
bull Connected live B-2 on ramp
at Whiteman AFB MO an
AWACS HITL at Tinker AFB
OK within a distributed C2
environment
IMPACT-- Cost Savings and Better
Product
bull Early testing led to early identification
and correction of Link 16 interoperability
issues
bull No range or flying costs
Army
Air Force
Navy
Marines
Joint
Industry
As of 14 Sep 2009
Tinker AFB AWACS
Whiteman B-2
Eglin 46TS
WPAFB SIMAF
JMETC
Langley GCIC
Army
Air Force
Navy
Marines
Joint
Industry
As of 14 Sep 2009
Tinker AFB AWACS
Whiteman B-2
Eglin 46TS
WPAFB SIMAF
JMETC
Langley GCIC
Post-Milestone C ndash Developmental Test
B-2 Spirit ICE Data Link Test
Sample of Measures
Timeliness and accuracy
of message exchanges
bull Joint Urgent Operational Need
bull Integration of BACN payload
onto multiple platforms for
solution to net-centric warfare
challenges of
bull Beyond line-of-sight comm
bull Relay bridge and range
extension for ground forces
and supporting aircraft
bull Distributed Testing included
Live-fly DT and Operational
Utility Evaluation Fall of 2010
IMPACT -- Cost Savings and
Better Product bullEfficient integration of DT and OT
bullAll required assets not available on-site
bullDistributed Testing saved ldquo$12Mrdquo
bullUrgent capability fielded
12
Responsive TampE for Rapid Fielding
Battlefield Airborne Communication Node
(BACN)
Challenges We Are Overcoming Today
Challenge
bull Site Interconnectivity
bull No common enabling
transport architecture
bull Inconsistent Tools
bull Test Participant
Collaboration
bull Analysis of Disparately
Stored Data
Overcome By
JMETC Network
TENA Middleware
JMETC ldquoBest of Breedrdquo
Process
Reuse Repository
Data Management Solutions
JMETC Joint Mission Environment Test Capability
TENA Test and Training Enabling Architecture
A TRMC Perspective
We Need a Common Vision for
Test and Training Infrastructure
Mr Derrick Hinton Principal Deputy Director Test Resource Management Center (TRMC)
Deputy Director Joint Investment Programs amp Policy (JIPP)
Office of the Undersecretary of Defense for Acquisition Technology amp Logistics (OUSD[ATampL])
Annual ITEA Test Instrumentation Workshop XIV
Test and Training Enterprise
12 May 2010
2010 2030
Individual Systems WHAT do we test Mission-Level
amp Systems of Systems Capabilities
Events WHEN do we test Continuously
Decision-Makers For WHOM do we test Decision-Makers
amp Warfighters
Agile
Streamlined
Affordable
Continuous
Goal Distributed Integrated Testing and Training
The DoD TampE process must evolve to be
Test Resource Management Center
Future TampE Process
15
Aspects of the 2030 Common
Vision
bull Test amp Training bull Real-Time TTP SOP Development to improve SoS effectiveness and
suitability (limited)
bull ldquoBlack boxrdquo and ldquowhite boxrdquo testing done in parallel
including
bull System functionality
bull System effectiveness
bull System of Systems effectiveness via mission threads in an
operationally relevant environment
bull Fewer test runsconfigurations with more meaningful
collected data
bull Real-Time Data Analysis amp Automated Reporting
OBJECTIVE Near Real-Time Test-Fix-Test 16
Some Challenges to Achieving
2030 Vision
bull Authoritative Joint Mission threads
bull Mission level measures
bull Thorough but Streamlined Information Assurance
Procedures
bull Enterprise-level Multi-Level Security solutions
bull Embedded Encryption on acquisition systems
bull RF over the wire (network)
bull Near Real Time Test-Fix-Test
bull Data Management
bull Embedded amp Unobtrusive Instrumentation
bull World wide wireless
bull Extremely efficient usage of the frequency spectrum 17
Examples Where the Vision Is Starting
to Come Together
bull Joint Integrated Air amp Missile Defense Organization
(JIAMDO) Correlation Decorrelation Interoperability Test
(CDIT) is reducing analysis cycle from 3 months to less
than a month
bull Changes to test configuration between test runs (~30 minutes)
bull Overnight AdditionSubtraction of test runs depending on issues
discovered and data analysis results
bull Real-time impact to test configuration and number of test runs
bull Emphasis on evaluators and tools for detailed data analysis
bull Air-Ground Integrated Layer Exploration (AGILE) bull AGILE provides an LVC capability for acquisition amp testing
bull Analytical Rigor provides decision quality data for decision support
bull Efficient amp collaborative use of Joint analytical resources (in
addition to service capabilities)
bull Sustains and evolves collaborative environment for future use 18
Some TRMC Investments for
Achieving Common Vision
bull Central TampE Investment Program (CTEIP)
bull Test and Training Enabling Architecture (TENA)
bull Multi-Level Security-Joint Coalition Network Environment (MLS-
JCNE)
bull InterTEC
bull Result JITC-certified instrumentation usable in all aspects of TampE
bull Common Range Integrated Instrumentation System (CRIIS)
bull TampE SampT Program
bull TENA in a Resource Constrained Environment (TRCE)
bull NST Evaluation Capability Model (NECM)
bull JMETC Program
bull DoD corporate approach for distributed testing 19
What Can You Do to Be Ready for
2030
bull Maintain Information Assurance agreements for all facility
systems
bull Streamline data collection amp analysis at your facility
bull Get comfortable with DoDAF views
bull Look across the RDTampE the community to find common
problems that can be addressed with similar solutions
bull TENA-enable your instrumentation systems and test
command amp control
bull Get involved share your requirements with your Service
representatives and in TRMC users meetings
20
JMETC Users Group
bull Purpose is to focus on technical requirements and solutions relevant to current and future Distributed Testing needs
bull Technical and Management level representatives identify core infrastructure requirements and most importantly resolve issues
bull Quarterly meetings of 250-300 JMETC customers acquisition programs test events ranges LVC sites tools and network providers
21
Last JMETC Users Group Meeting bull June 2011 bull Location Mid-East Coast bull Tracks
bull User Requirements bull Information Assurance
Security bull Data Management bull InterTEC (Current amp Planned) bull Networking
bull An established forum for the Distributed Test Community to
o Identify core infrastructure requirements and use cases
o Identify investigate amp resolve issues
o Identify opportunities to collaborate
o Discuss available solutions tools and techniques
o Share lessons learned
Summary
bull We are becoming more efficient in how we execute TampE
bull We must get the biggest and best ldquobang for the buckrdquo that we can
bull TRMC and Services continue TampE investments to enable realization of the 2030 Vision
bull Align TampE (OSD and Services) investments to mitigate test risk areas
bull The Future of TampE will include Distributed Integrated Testing and Training and will enable near real time Test-Fix-Test
22
JMETC Program Points of Contact
23
JMETC Program Manager Chip Ferguson chipfergusonosdmil 703-601-5274
JMETC Principal Deputy PM Bruce Bailey brucebaileyosdmil 703-601-5208
JMETC Lead Operations Planning Marty Arnwine martemasarnwineosdmil 703-601-5215
JMETC Senior Technical Advisor George Rumford georgerumfordosdmil
703-601-5233
JMETC Lead Systems Engineer Ryan Norman ryannormanosdmil
703-601-5277
JMETC Lead Network Engineer Arjuna ldquoAJrdquo Pathmanathan ArjunaPathmanathanosdmil
703-601-5214
JMETC Website wwwjmetcorg
Backup Slides
25
FY10 JMETC Events
Customer Event Record Event Dates
Air Force JEFX 10-1Spirit Ice (B2 Data Link Test) October ndash November 2009
Navy BAMS LVC DE October ndash December 2009
Air Force Battlefield Airborne Comm Node (BACN) JUON
(DTOT)
November 2009 to September 2010
Air Force Agile Fire 10-2 January 2010
Air Force JEFX 10-23 FebruaryApril 2010
Joint Joint Surface Warfare JCTD February to September 2010
JS J8JIAMDO Joint Sensor Integration April to September 2010
Air Force B1-B Fully Integrated Data Link Testing April 2010
JFCOM J8489 (TEST WEEK) JCAS Distributed Test June 2010
JIAMDO (Navy Lead) CorrelationDecorrelation Interoperability Test
(CDIT) Integration Events (Continuous)
July to September 2010
Army (Lead) UAS in the National Airspace July to September 2010
Air Force Agile Fire 10-3 August 2010
Joint JITC Joint Interoperability Tests JIT 10-3 amp 11-1
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Joint Strike Fighter (JSF) JIAMDOJoint Track Manager
Brigade Combat Team (BCT)
Modernization
Multi-Function Adv Data Link (MADL) Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run 25
26
JMETC FY10 Accomplishments
JMETC Accomplishments
Supported 88 distinct customer test activities
Expanded network from 38 to 57 sites
ATIN and JTDL Networks transitioned to JMETC
Upgraded JMETC support applications and utilities to TENA R6
DIACAP Tiger Team report completed and recommendations being executed
Enhanced JMETC services and capabilities provided by leveraging InterTEC Services and Industry
Reuse Repository usability improvements
FY10 Example JMETC Customers
Joint Integrated Air amp Missile Defense Organization (JIAMDO)
Broad Area Communications Node (BACN) JUON
B1-B
Broad Area Maritime Surveillance System (BAMS)
Air-Ground Integrated Layer Exploration (AGILE)
Joint Interoperability Test Command (JITC)
Selected Benefits to the DoD
Integrated DT amp OT on a Joint Urgent Operational Need for the warfighter
Maximized usage of theater assets during limited maintenance windows
Improved Joint track information sharing to ensure interoperability of systems in theater operations
Coalition exchange and examination of real-time air picture data
Identification of Air Force Initiatives ready for warfighter transition
Investigated tactical UAS deployment in the National Airspace
Employment of Net-Enabled Weapons
JCAS immediate request amp end-to-end processes ldquoas-isrdquo characterization
Determined distributed system components were not ready for full live integration testing
Executed testing to support system-of-system interoperability certification
27
Customer Event Record Event Dates
Joint JIAMDO CorrelationDecorrelation Interoperability Test (CDIT)
Integration Events (Continuous) OCONUS October 2010
Navy UAS in NAS Runs for Record October 2010
Internal InterTEC Spiral 3 Systems Acceptance Test October - November 2010
Joint JITC Joint Interoperability Tests JIT 11-12345 (Continuous) October 2010 ndash September 2011
Air Force B1-B Fully Integrated Data Link Testing November 2010
Air Force AGILE Fire Phase III JEFX 2011 December 2010 - February 2011
Joint JTRS JPO -- JTRS Ground Mobile Radio January 2011
Navy Broad Area Maritime Surveillance (BAMS) (Continuous) January ndash September 2011
Joint Joint Track Manager Capability Demonstration (Continuous) January ndash September 2011
Joint JSF Initial MampS Interoperability (Continuous) February ndash March 2011
Air Force JSTARS Interoperability Test May 2011
Joint JS J8JIAMDO Joint Sensor Integration June ndash August 2011
FY11 JMETC Events
(More to Come)
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Global Hawk GATOR
Brigade Combat Team (BCT)
Modernization F-22 FY12 Testing Planned Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run
28
JMETC Connectivity
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca (2) JITC JTDL
Redstone (7) SMDC
RTC DTCC GMAN
SED Patriot THAAD FAAD SIV
Charleston (2)IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB SIMAF
As of 16 Mar 2011
0
10
20
30
40
50
60
70
FY06 FY07 FY08 FY09 FY10 FY11
JMETC Site Expansion
Sites
Dedicated trusted connectivity on SDREN (part of the GIG)
Encrypted for Secret ndash System High
DISA-registered IP address space
Active monitoring of network performance
Capable of supporting multiple simultaneous test events Bethpage NGC BAMS
Whiteman B-2
Eglin AFB (4)AOC DTF GWEF KHILS
Hurlburt Field C2DAC
Sites in Gulf Range
Functional Sites 61
New Sites Planned 8
Connection Points to Other Networks 5
Ft Greely CRTC
Site in Alaska
MHPCC
PMRF Bldg 105
Sites in Hawaii
Camp PendletonMCTSSA
China Lake (3)AV-8B FA-18IBAR
Edwards Ridley
Corona NSWC
El Segundo NGC B-2
Sites in SoCal
Point Loma (2)RLBTSSSC-PAC 59140 SSC-PAC HAIPE
Point Mugu (2)ITEC EA-6B
West Agg Rtr
Rancho Bernardo NGC BAMS
Pax River (5)ESTEL E2CD MCLACETEF SAIL ATR JMETC SYSCONEast Agg Rtr
Aberdeen ACCN
Langley (2) C-GIIF TDLITC
JFCOM JSIC
Wallops Island (2) SCSC SSDS
DISA Sky 7
Pentagon WARCAP
Dahlgren (2) CEDLIWSL
Dam Neck CDSA
Newport News NGC VASCIC
Sites in MD DC VA
Norfolk COMOPTEVFOR
McLean MITRE NCEL
Ft Huachuca JTRS
Bethpage NGC BCN
How an Air Force Customer Sees
Distributed Connectivity
Hardware-in-the-Loop (HWIL) Availability
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca JITC
Redstone (3) DTCC GMAN SED
Charleston (2) IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB
SIMAF
Bethpage NG BAMS
Whiteman B-2
MHPCC
PMRF Bldg 105
Sites in Hawaii
All linked by JMETC
29
Challenges Explained in Detail
30
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
bull Joint Urgent Operational Need
bull Integration of BACN payload
onto multiple platforms for
solution to net-centric warfare
challenges of
bull Beyond line-of-sight comm
bull Relay bridge and range
extension for ground forces
and supporting aircraft
bull Distributed Testing included
Live-fly DT and Operational
Utility Evaluation Fall of 2010
IMPACT -- Cost Savings and
Better Product bullEfficient integration of DT and OT
bullAll required assets not available on-site
bullDistributed Testing saved ldquo$12Mrdquo
bullUrgent capability fielded
12
Responsive TampE for Rapid Fielding
Battlefield Airborne Communication Node
(BACN)
Challenges We Are Overcoming Today
Challenge
bull Site Interconnectivity
bull No common enabling
transport architecture
bull Inconsistent Tools
bull Test Participant
Collaboration
bull Analysis of Disparately
Stored Data
Overcome By
JMETC Network
TENA Middleware
JMETC ldquoBest of Breedrdquo
Process
Reuse Repository
Data Management Solutions
JMETC Joint Mission Environment Test Capability
TENA Test and Training Enabling Architecture
A TRMC Perspective
We Need a Common Vision for
Test and Training Infrastructure
Mr Derrick Hinton Principal Deputy Director Test Resource Management Center (TRMC)
Deputy Director Joint Investment Programs amp Policy (JIPP)
Office of the Undersecretary of Defense for Acquisition Technology amp Logistics (OUSD[ATampL])
Annual ITEA Test Instrumentation Workshop XIV
Test and Training Enterprise
12 May 2010
2010 2030
Individual Systems WHAT do we test Mission-Level
amp Systems of Systems Capabilities
Events WHEN do we test Continuously
Decision-Makers For WHOM do we test Decision-Makers
amp Warfighters
Agile
Streamlined
Affordable
Continuous
Goal Distributed Integrated Testing and Training
The DoD TampE process must evolve to be
Test Resource Management Center
Future TampE Process
15
Aspects of the 2030 Common
Vision
bull Test amp Training bull Real-Time TTP SOP Development to improve SoS effectiveness and
suitability (limited)
bull ldquoBlack boxrdquo and ldquowhite boxrdquo testing done in parallel
including
bull System functionality
bull System effectiveness
bull System of Systems effectiveness via mission threads in an
operationally relevant environment
bull Fewer test runsconfigurations with more meaningful
collected data
bull Real-Time Data Analysis amp Automated Reporting
OBJECTIVE Near Real-Time Test-Fix-Test 16
Some Challenges to Achieving
2030 Vision
bull Authoritative Joint Mission threads
bull Mission level measures
bull Thorough but Streamlined Information Assurance
Procedures
bull Enterprise-level Multi-Level Security solutions
bull Embedded Encryption on acquisition systems
bull RF over the wire (network)
bull Near Real Time Test-Fix-Test
bull Data Management
bull Embedded amp Unobtrusive Instrumentation
bull World wide wireless
bull Extremely efficient usage of the frequency spectrum 17
Examples Where the Vision Is Starting
to Come Together
bull Joint Integrated Air amp Missile Defense Organization
(JIAMDO) Correlation Decorrelation Interoperability Test
(CDIT) is reducing analysis cycle from 3 months to less
than a month
bull Changes to test configuration between test runs (~30 minutes)
bull Overnight AdditionSubtraction of test runs depending on issues
discovered and data analysis results
bull Real-time impact to test configuration and number of test runs
bull Emphasis on evaluators and tools for detailed data analysis
bull Air-Ground Integrated Layer Exploration (AGILE) bull AGILE provides an LVC capability for acquisition amp testing
bull Analytical Rigor provides decision quality data for decision support
bull Efficient amp collaborative use of Joint analytical resources (in
addition to service capabilities)
bull Sustains and evolves collaborative environment for future use 18
Some TRMC Investments for
Achieving Common Vision
bull Central TampE Investment Program (CTEIP)
bull Test and Training Enabling Architecture (TENA)
bull Multi-Level Security-Joint Coalition Network Environment (MLS-
JCNE)
bull InterTEC
bull Result JITC-certified instrumentation usable in all aspects of TampE
bull Common Range Integrated Instrumentation System (CRIIS)
bull TampE SampT Program
bull TENA in a Resource Constrained Environment (TRCE)
bull NST Evaluation Capability Model (NECM)
bull JMETC Program
bull DoD corporate approach for distributed testing 19
What Can You Do to Be Ready for
2030
bull Maintain Information Assurance agreements for all facility
systems
bull Streamline data collection amp analysis at your facility
bull Get comfortable with DoDAF views
bull Look across the RDTampE the community to find common
problems that can be addressed with similar solutions
bull TENA-enable your instrumentation systems and test
command amp control
bull Get involved share your requirements with your Service
representatives and in TRMC users meetings
20
JMETC Users Group
bull Purpose is to focus on technical requirements and solutions relevant to current and future Distributed Testing needs
bull Technical and Management level representatives identify core infrastructure requirements and most importantly resolve issues
bull Quarterly meetings of 250-300 JMETC customers acquisition programs test events ranges LVC sites tools and network providers
21
Last JMETC Users Group Meeting bull June 2011 bull Location Mid-East Coast bull Tracks
bull User Requirements bull Information Assurance
Security bull Data Management bull InterTEC (Current amp Planned) bull Networking
bull An established forum for the Distributed Test Community to
o Identify core infrastructure requirements and use cases
o Identify investigate amp resolve issues
o Identify opportunities to collaborate
o Discuss available solutions tools and techniques
o Share lessons learned
Summary
bull We are becoming more efficient in how we execute TampE
bull We must get the biggest and best ldquobang for the buckrdquo that we can
bull TRMC and Services continue TampE investments to enable realization of the 2030 Vision
bull Align TampE (OSD and Services) investments to mitigate test risk areas
bull The Future of TampE will include Distributed Integrated Testing and Training and will enable near real time Test-Fix-Test
22
JMETC Program Points of Contact
23
JMETC Program Manager Chip Ferguson chipfergusonosdmil 703-601-5274
JMETC Principal Deputy PM Bruce Bailey brucebaileyosdmil 703-601-5208
JMETC Lead Operations Planning Marty Arnwine martemasarnwineosdmil 703-601-5215
JMETC Senior Technical Advisor George Rumford georgerumfordosdmil
703-601-5233
JMETC Lead Systems Engineer Ryan Norman ryannormanosdmil
703-601-5277
JMETC Lead Network Engineer Arjuna ldquoAJrdquo Pathmanathan ArjunaPathmanathanosdmil
703-601-5214
JMETC Website wwwjmetcorg
Backup Slides
25
FY10 JMETC Events
Customer Event Record Event Dates
Air Force JEFX 10-1Spirit Ice (B2 Data Link Test) October ndash November 2009
Navy BAMS LVC DE October ndash December 2009
Air Force Battlefield Airborne Comm Node (BACN) JUON
(DTOT)
November 2009 to September 2010
Air Force Agile Fire 10-2 January 2010
Air Force JEFX 10-23 FebruaryApril 2010
Joint Joint Surface Warfare JCTD February to September 2010
JS J8JIAMDO Joint Sensor Integration April to September 2010
Air Force B1-B Fully Integrated Data Link Testing April 2010
JFCOM J8489 (TEST WEEK) JCAS Distributed Test June 2010
JIAMDO (Navy Lead) CorrelationDecorrelation Interoperability Test
(CDIT) Integration Events (Continuous)
July to September 2010
Army (Lead) UAS in the National Airspace July to September 2010
Air Force Agile Fire 10-3 August 2010
Joint JITC Joint Interoperability Tests JIT 10-3 amp 11-1
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Joint Strike Fighter (JSF) JIAMDOJoint Track Manager
Brigade Combat Team (BCT)
Modernization
Multi-Function Adv Data Link (MADL) Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run 25
26
JMETC FY10 Accomplishments
JMETC Accomplishments
Supported 88 distinct customer test activities
Expanded network from 38 to 57 sites
ATIN and JTDL Networks transitioned to JMETC
Upgraded JMETC support applications and utilities to TENA R6
DIACAP Tiger Team report completed and recommendations being executed
Enhanced JMETC services and capabilities provided by leveraging InterTEC Services and Industry
Reuse Repository usability improvements
FY10 Example JMETC Customers
Joint Integrated Air amp Missile Defense Organization (JIAMDO)
Broad Area Communications Node (BACN) JUON
B1-B
Broad Area Maritime Surveillance System (BAMS)
Air-Ground Integrated Layer Exploration (AGILE)
Joint Interoperability Test Command (JITC)
Selected Benefits to the DoD
Integrated DT amp OT on a Joint Urgent Operational Need for the warfighter
Maximized usage of theater assets during limited maintenance windows
Improved Joint track information sharing to ensure interoperability of systems in theater operations
Coalition exchange and examination of real-time air picture data
Identification of Air Force Initiatives ready for warfighter transition
Investigated tactical UAS deployment in the National Airspace
Employment of Net-Enabled Weapons
JCAS immediate request amp end-to-end processes ldquoas-isrdquo characterization
Determined distributed system components were not ready for full live integration testing
Executed testing to support system-of-system interoperability certification
27
Customer Event Record Event Dates
Joint JIAMDO CorrelationDecorrelation Interoperability Test (CDIT)
Integration Events (Continuous) OCONUS October 2010
Navy UAS in NAS Runs for Record October 2010
Internal InterTEC Spiral 3 Systems Acceptance Test October - November 2010
Joint JITC Joint Interoperability Tests JIT 11-12345 (Continuous) October 2010 ndash September 2011
Air Force B1-B Fully Integrated Data Link Testing November 2010
Air Force AGILE Fire Phase III JEFX 2011 December 2010 - February 2011
Joint JTRS JPO -- JTRS Ground Mobile Radio January 2011
Navy Broad Area Maritime Surveillance (BAMS) (Continuous) January ndash September 2011
Joint Joint Track Manager Capability Demonstration (Continuous) January ndash September 2011
Joint JSF Initial MampS Interoperability (Continuous) February ndash March 2011
Air Force JSTARS Interoperability Test May 2011
Joint JS J8JIAMDO Joint Sensor Integration June ndash August 2011
FY11 JMETC Events
(More to Come)
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Global Hawk GATOR
Brigade Combat Team (BCT)
Modernization F-22 FY12 Testing Planned Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run
28
JMETC Connectivity
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca (2) JITC JTDL
Redstone (7) SMDC
RTC DTCC GMAN
SED Patriot THAAD FAAD SIV
Charleston (2)IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB SIMAF
As of 16 Mar 2011
0
10
20
30
40
50
60
70
FY06 FY07 FY08 FY09 FY10 FY11
JMETC Site Expansion
Sites
Dedicated trusted connectivity on SDREN (part of the GIG)
Encrypted for Secret ndash System High
DISA-registered IP address space
Active monitoring of network performance
Capable of supporting multiple simultaneous test events Bethpage NGC BAMS
Whiteman B-2
Eglin AFB (4)AOC DTF GWEF KHILS
Hurlburt Field C2DAC
Sites in Gulf Range
Functional Sites 61
New Sites Planned 8
Connection Points to Other Networks 5
Ft Greely CRTC
Site in Alaska
MHPCC
PMRF Bldg 105
Sites in Hawaii
Camp PendletonMCTSSA
China Lake (3)AV-8B FA-18IBAR
Edwards Ridley
Corona NSWC
El Segundo NGC B-2
Sites in SoCal
Point Loma (2)RLBTSSSC-PAC 59140 SSC-PAC HAIPE
Point Mugu (2)ITEC EA-6B
West Agg Rtr
Rancho Bernardo NGC BAMS
Pax River (5)ESTEL E2CD MCLACETEF SAIL ATR JMETC SYSCONEast Agg Rtr
Aberdeen ACCN
Langley (2) C-GIIF TDLITC
JFCOM JSIC
Wallops Island (2) SCSC SSDS
DISA Sky 7
Pentagon WARCAP
Dahlgren (2) CEDLIWSL
Dam Neck CDSA
Newport News NGC VASCIC
Sites in MD DC VA
Norfolk COMOPTEVFOR
McLean MITRE NCEL
Ft Huachuca JTRS
Bethpage NGC BCN
How an Air Force Customer Sees
Distributed Connectivity
Hardware-in-the-Loop (HWIL) Availability
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca JITC
Redstone (3) DTCC GMAN SED
Charleston (2) IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB
SIMAF
Bethpage NG BAMS
Whiteman B-2
MHPCC
PMRF Bldg 105
Sites in Hawaii
All linked by JMETC
29
Challenges Explained in Detail
30
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
Challenges We Are Overcoming Today
Challenge
bull Site Interconnectivity
bull No common enabling
transport architecture
bull Inconsistent Tools
bull Test Participant
Collaboration
bull Analysis of Disparately
Stored Data
Overcome By
JMETC Network
TENA Middleware
JMETC ldquoBest of Breedrdquo
Process
Reuse Repository
Data Management Solutions
JMETC Joint Mission Environment Test Capability
TENA Test and Training Enabling Architecture
A TRMC Perspective
We Need a Common Vision for
Test and Training Infrastructure
Mr Derrick Hinton Principal Deputy Director Test Resource Management Center (TRMC)
Deputy Director Joint Investment Programs amp Policy (JIPP)
Office of the Undersecretary of Defense for Acquisition Technology amp Logistics (OUSD[ATampL])
Annual ITEA Test Instrumentation Workshop XIV
Test and Training Enterprise
12 May 2010
2010 2030
Individual Systems WHAT do we test Mission-Level
amp Systems of Systems Capabilities
Events WHEN do we test Continuously
Decision-Makers For WHOM do we test Decision-Makers
amp Warfighters
Agile
Streamlined
Affordable
Continuous
Goal Distributed Integrated Testing and Training
The DoD TampE process must evolve to be
Test Resource Management Center
Future TampE Process
15
Aspects of the 2030 Common
Vision
bull Test amp Training bull Real-Time TTP SOP Development to improve SoS effectiveness and
suitability (limited)
bull ldquoBlack boxrdquo and ldquowhite boxrdquo testing done in parallel
including
bull System functionality
bull System effectiveness
bull System of Systems effectiveness via mission threads in an
operationally relevant environment
bull Fewer test runsconfigurations with more meaningful
collected data
bull Real-Time Data Analysis amp Automated Reporting
OBJECTIVE Near Real-Time Test-Fix-Test 16
Some Challenges to Achieving
2030 Vision
bull Authoritative Joint Mission threads
bull Mission level measures
bull Thorough but Streamlined Information Assurance
Procedures
bull Enterprise-level Multi-Level Security solutions
bull Embedded Encryption on acquisition systems
bull RF over the wire (network)
bull Near Real Time Test-Fix-Test
bull Data Management
bull Embedded amp Unobtrusive Instrumentation
bull World wide wireless
bull Extremely efficient usage of the frequency spectrum 17
Examples Where the Vision Is Starting
to Come Together
bull Joint Integrated Air amp Missile Defense Organization
(JIAMDO) Correlation Decorrelation Interoperability Test
(CDIT) is reducing analysis cycle from 3 months to less
than a month
bull Changes to test configuration between test runs (~30 minutes)
bull Overnight AdditionSubtraction of test runs depending on issues
discovered and data analysis results
bull Real-time impact to test configuration and number of test runs
bull Emphasis on evaluators and tools for detailed data analysis
bull Air-Ground Integrated Layer Exploration (AGILE) bull AGILE provides an LVC capability for acquisition amp testing
bull Analytical Rigor provides decision quality data for decision support
bull Efficient amp collaborative use of Joint analytical resources (in
addition to service capabilities)
bull Sustains and evolves collaborative environment for future use 18
Some TRMC Investments for
Achieving Common Vision
bull Central TampE Investment Program (CTEIP)
bull Test and Training Enabling Architecture (TENA)
bull Multi-Level Security-Joint Coalition Network Environment (MLS-
JCNE)
bull InterTEC
bull Result JITC-certified instrumentation usable in all aspects of TampE
bull Common Range Integrated Instrumentation System (CRIIS)
bull TampE SampT Program
bull TENA in a Resource Constrained Environment (TRCE)
bull NST Evaluation Capability Model (NECM)
bull JMETC Program
bull DoD corporate approach for distributed testing 19
What Can You Do to Be Ready for
2030
bull Maintain Information Assurance agreements for all facility
systems
bull Streamline data collection amp analysis at your facility
bull Get comfortable with DoDAF views
bull Look across the RDTampE the community to find common
problems that can be addressed with similar solutions
bull TENA-enable your instrumentation systems and test
command amp control
bull Get involved share your requirements with your Service
representatives and in TRMC users meetings
20
JMETC Users Group
bull Purpose is to focus on technical requirements and solutions relevant to current and future Distributed Testing needs
bull Technical and Management level representatives identify core infrastructure requirements and most importantly resolve issues
bull Quarterly meetings of 250-300 JMETC customers acquisition programs test events ranges LVC sites tools and network providers
21
Last JMETC Users Group Meeting bull June 2011 bull Location Mid-East Coast bull Tracks
bull User Requirements bull Information Assurance
Security bull Data Management bull InterTEC (Current amp Planned) bull Networking
bull An established forum for the Distributed Test Community to
o Identify core infrastructure requirements and use cases
o Identify investigate amp resolve issues
o Identify opportunities to collaborate
o Discuss available solutions tools and techniques
o Share lessons learned
Summary
bull We are becoming more efficient in how we execute TampE
bull We must get the biggest and best ldquobang for the buckrdquo that we can
bull TRMC and Services continue TampE investments to enable realization of the 2030 Vision
bull Align TampE (OSD and Services) investments to mitigate test risk areas
bull The Future of TampE will include Distributed Integrated Testing and Training and will enable near real time Test-Fix-Test
22
JMETC Program Points of Contact
23
JMETC Program Manager Chip Ferguson chipfergusonosdmil 703-601-5274
JMETC Principal Deputy PM Bruce Bailey brucebaileyosdmil 703-601-5208
JMETC Lead Operations Planning Marty Arnwine martemasarnwineosdmil 703-601-5215
JMETC Senior Technical Advisor George Rumford georgerumfordosdmil
703-601-5233
JMETC Lead Systems Engineer Ryan Norman ryannormanosdmil
703-601-5277
JMETC Lead Network Engineer Arjuna ldquoAJrdquo Pathmanathan ArjunaPathmanathanosdmil
703-601-5214
JMETC Website wwwjmetcorg
Backup Slides
25
FY10 JMETC Events
Customer Event Record Event Dates
Air Force JEFX 10-1Spirit Ice (B2 Data Link Test) October ndash November 2009
Navy BAMS LVC DE October ndash December 2009
Air Force Battlefield Airborne Comm Node (BACN) JUON
(DTOT)
November 2009 to September 2010
Air Force Agile Fire 10-2 January 2010
Air Force JEFX 10-23 FebruaryApril 2010
Joint Joint Surface Warfare JCTD February to September 2010
JS J8JIAMDO Joint Sensor Integration April to September 2010
Air Force B1-B Fully Integrated Data Link Testing April 2010
JFCOM J8489 (TEST WEEK) JCAS Distributed Test June 2010
JIAMDO (Navy Lead) CorrelationDecorrelation Interoperability Test
(CDIT) Integration Events (Continuous)
July to September 2010
Army (Lead) UAS in the National Airspace July to September 2010
Air Force Agile Fire 10-3 August 2010
Joint JITC Joint Interoperability Tests JIT 10-3 amp 11-1
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Joint Strike Fighter (JSF) JIAMDOJoint Track Manager
Brigade Combat Team (BCT)
Modernization
Multi-Function Adv Data Link (MADL) Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run 25
26
JMETC FY10 Accomplishments
JMETC Accomplishments
Supported 88 distinct customer test activities
Expanded network from 38 to 57 sites
ATIN and JTDL Networks transitioned to JMETC
Upgraded JMETC support applications and utilities to TENA R6
DIACAP Tiger Team report completed and recommendations being executed
Enhanced JMETC services and capabilities provided by leveraging InterTEC Services and Industry
Reuse Repository usability improvements
FY10 Example JMETC Customers
Joint Integrated Air amp Missile Defense Organization (JIAMDO)
Broad Area Communications Node (BACN) JUON
B1-B
Broad Area Maritime Surveillance System (BAMS)
Air-Ground Integrated Layer Exploration (AGILE)
Joint Interoperability Test Command (JITC)
Selected Benefits to the DoD
Integrated DT amp OT on a Joint Urgent Operational Need for the warfighter
Maximized usage of theater assets during limited maintenance windows
Improved Joint track information sharing to ensure interoperability of systems in theater operations
Coalition exchange and examination of real-time air picture data
Identification of Air Force Initiatives ready for warfighter transition
Investigated tactical UAS deployment in the National Airspace
Employment of Net-Enabled Weapons
JCAS immediate request amp end-to-end processes ldquoas-isrdquo characterization
Determined distributed system components were not ready for full live integration testing
Executed testing to support system-of-system interoperability certification
27
Customer Event Record Event Dates
Joint JIAMDO CorrelationDecorrelation Interoperability Test (CDIT)
Integration Events (Continuous) OCONUS October 2010
Navy UAS in NAS Runs for Record October 2010
Internal InterTEC Spiral 3 Systems Acceptance Test October - November 2010
Joint JITC Joint Interoperability Tests JIT 11-12345 (Continuous) October 2010 ndash September 2011
Air Force B1-B Fully Integrated Data Link Testing November 2010
Air Force AGILE Fire Phase III JEFX 2011 December 2010 - February 2011
Joint JTRS JPO -- JTRS Ground Mobile Radio January 2011
Navy Broad Area Maritime Surveillance (BAMS) (Continuous) January ndash September 2011
Joint Joint Track Manager Capability Demonstration (Continuous) January ndash September 2011
Joint JSF Initial MampS Interoperability (Continuous) February ndash March 2011
Air Force JSTARS Interoperability Test May 2011
Joint JS J8JIAMDO Joint Sensor Integration June ndash August 2011
FY11 JMETC Events
(More to Come)
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Global Hawk GATOR
Brigade Combat Team (BCT)
Modernization F-22 FY12 Testing Planned Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run
28
JMETC Connectivity
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca (2) JITC JTDL
Redstone (7) SMDC
RTC DTCC GMAN
SED Patriot THAAD FAAD SIV
Charleston (2)IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB SIMAF
As of 16 Mar 2011
0
10
20
30
40
50
60
70
FY06 FY07 FY08 FY09 FY10 FY11
JMETC Site Expansion
Sites
Dedicated trusted connectivity on SDREN (part of the GIG)
Encrypted for Secret ndash System High
DISA-registered IP address space
Active monitoring of network performance
Capable of supporting multiple simultaneous test events Bethpage NGC BAMS
Whiteman B-2
Eglin AFB (4)AOC DTF GWEF KHILS
Hurlburt Field C2DAC
Sites in Gulf Range
Functional Sites 61
New Sites Planned 8
Connection Points to Other Networks 5
Ft Greely CRTC
Site in Alaska
MHPCC
PMRF Bldg 105
Sites in Hawaii
Camp PendletonMCTSSA
China Lake (3)AV-8B FA-18IBAR
Edwards Ridley
Corona NSWC
El Segundo NGC B-2
Sites in SoCal
Point Loma (2)RLBTSSSC-PAC 59140 SSC-PAC HAIPE
Point Mugu (2)ITEC EA-6B
West Agg Rtr
Rancho Bernardo NGC BAMS
Pax River (5)ESTEL E2CD MCLACETEF SAIL ATR JMETC SYSCONEast Agg Rtr
Aberdeen ACCN
Langley (2) C-GIIF TDLITC
JFCOM JSIC
Wallops Island (2) SCSC SSDS
DISA Sky 7
Pentagon WARCAP
Dahlgren (2) CEDLIWSL
Dam Neck CDSA
Newport News NGC VASCIC
Sites in MD DC VA
Norfolk COMOPTEVFOR
McLean MITRE NCEL
Ft Huachuca JTRS
Bethpage NGC BCN
How an Air Force Customer Sees
Distributed Connectivity
Hardware-in-the-Loop (HWIL) Availability
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca JITC
Redstone (3) DTCC GMAN SED
Charleston (2) IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB
SIMAF
Bethpage NG BAMS
Whiteman B-2
MHPCC
PMRF Bldg 105
Sites in Hawaii
All linked by JMETC
29
Challenges Explained in Detail
30
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
A TRMC Perspective
We Need a Common Vision for
Test and Training Infrastructure
Mr Derrick Hinton Principal Deputy Director Test Resource Management Center (TRMC)
Deputy Director Joint Investment Programs amp Policy (JIPP)
Office of the Undersecretary of Defense for Acquisition Technology amp Logistics (OUSD[ATampL])
Annual ITEA Test Instrumentation Workshop XIV
Test and Training Enterprise
12 May 2010
2010 2030
Individual Systems WHAT do we test Mission-Level
amp Systems of Systems Capabilities
Events WHEN do we test Continuously
Decision-Makers For WHOM do we test Decision-Makers
amp Warfighters
Agile
Streamlined
Affordable
Continuous
Goal Distributed Integrated Testing and Training
The DoD TampE process must evolve to be
Test Resource Management Center
Future TampE Process
15
Aspects of the 2030 Common
Vision
bull Test amp Training bull Real-Time TTP SOP Development to improve SoS effectiveness and
suitability (limited)
bull ldquoBlack boxrdquo and ldquowhite boxrdquo testing done in parallel
including
bull System functionality
bull System effectiveness
bull System of Systems effectiveness via mission threads in an
operationally relevant environment
bull Fewer test runsconfigurations with more meaningful
collected data
bull Real-Time Data Analysis amp Automated Reporting
OBJECTIVE Near Real-Time Test-Fix-Test 16
Some Challenges to Achieving
2030 Vision
bull Authoritative Joint Mission threads
bull Mission level measures
bull Thorough but Streamlined Information Assurance
Procedures
bull Enterprise-level Multi-Level Security solutions
bull Embedded Encryption on acquisition systems
bull RF over the wire (network)
bull Near Real Time Test-Fix-Test
bull Data Management
bull Embedded amp Unobtrusive Instrumentation
bull World wide wireless
bull Extremely efficient usage of the frequency spectrum 17
Examples Where the Vision Is Starting
to Come Together
bull Joint Integrated Air amp Missile Defense Organization
(JIAMDO) Correlation Decorrelation Interoperability Test
(CDIT) is reducing analysis cycle from 3 months to less
than a month
bull Changes to test configuration between test runs (~30 minutes)
bull Overnight AdditionSubtraction of test runs depending on issues
discovered and data analysis results
bull Real-time impact to test configuration and number of test runs
bull Emphasis on evaluators and tools for detailed data analysis
bull Air-Ground Integrated Layer Exploration (AGILE) bull AGILE provides an LVC capability for acquisition amp testing
bull Analytical Rigor provides decision quality data for decision support
bull Efficient amp collaborative use of Joint analytical resources (in
addition to service capabilities)
bull Sustains and evolves collaborative environment for future use 18
Some TRMC Investments for
Achieving Common Vision
bull Central TampE Investment Program (CTEIP)
bull Test and Training Enabling Architecture (TENA)
bull Multi-Level Security-Joint Coalition Network Environment (MLS-
JCNE)
bull InterTEC
bull Result JITC-certified instrumentation usable in all aspects of TampE
bull Common Range Integrated Instrumentation System (CRIIS)
bull TampE SampT Program
bull TENA in a Resource Constrained Environment (TRCE)
bull NST Evaluation Capability Model (NECM)
bull JMETC Program
bull DoD corporate approach for distributed testing 19
What Can You Do to Be Ready for
2030
bull Maintain Information Assurance agreements for all facility
systems
bull Streamline data collection amp analysis at your facility
bull Get comfortable with DoDAF views
bull Look across the RDTampE the community to find common
problems that can be addressed with similar solutions
bull TENA-enable your instrumentation systems and test
command amp control
bull Get involved share your requirements with your Service
representatives and in TRMC users meetings
20
JMETC Users Group
bull Purpose is to focus on technical requirements and solutions relevant to current and future Distributed Testing needs
bull Technical and Management level representatives identify core infrastructure requirements and most importantly resolve issues
bull Quarterly meetings of 250-300 JMETC customers acquisition programs test events ranges LVC sites tools and network providers
21
Last JMETC Users Group Meeting bull June 2011 bull Location Mid-East Coast bull Tracks
bull User Requirements bull Information Assurance
Security bull Data Management bull InterTEC (Current amp Planned) bull Networking
bull An established forum for the Distributed Test Community to
o Identify core infrastructure requirements and use cases
o Identify investigate amp resolve issues
o Identify opportunities to collaborate
o Discuss available solutions tools and techniques
o Share lessons learned
Summary
bull We are becoming more efficient in how we execute TampE
bull We must get the biggest and best ldquobang for the buckrdquo that we can
bull TRMC and Services continue TampE investments to enable realization of the 2030 Vision
bull Align TampE (OSD and Services) investments to mitigate test risk areas
bull The Future of TampE will include Distributed Integrated Testing and Training and will enable near real time Test-Fix-Test
22
JMETC Program Points of Contact
23
JMETC Program Manager Chip Ferguson chipfergusonosdmil 703-601-5274
JMETC Principal Deputy PM Bruce Bailey brucebaileyosdmil 703-601-5208
JMETC Lead Operations Planning Marty Arnwine martemasarnwineosdmil 703-601-5215
JMETC Senior Technical Advisor George Rumford georgerumfordosdmil
703-601-5233
JMETC Lead Systems Engineer Ryan Norman ryannormanosdmil
703-601-5277
JMETC Lead Network Engineer Arjuna ldquoAJrdquo Pathmanathan ArjunaPathmanathanosdmil
703-601-5214
JMETC Website wwwjmetcorg
Backup Slides
25
FY10 JMETC Events
Customer Event Record Event Dates
Air Force JEFX 10-1Spirit Ice (B2 Data Link Test) October ndash November 2009
Navy BAMS LVC DE October ndash December 2009
Air Force Battlefield Airborne Comm Node (BACN) JUON
(DTOT)
November 2009 to September 2010
Air Force Agile Fire 10-2 January 2010
Air Force JEFX 10-23 FebruaryApril 2010
Joint Joint Surface Warfare JCTD February to September 2010
JS J8JIAMDO Joint Sensor Integration April to September 2010
Air Force B1-B Fully Integrated Data Link Testing April 2010
JFCOM J8489 (TEST WEEK) JCAS Distributed Test June 2010
JIAMDO (Navy Lead) CorrelationDecorrelation Interoperability Test
(CDIT) Integration Events (Continuous)
July to September 2010
Army (Lead) UAS in the National Airspace July to September 2010
Air Force Agile Fire 10-3 August 2010
Joint JITC Joint Interoperability Tests JIT 10-3 amp 11-1
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Joint Strike Fighter (JSF) JIAMDOJoint Track Manager
Brigade Combat Team (BCT)
Modernization
Multi-Function Adv Data Link (MADL) Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run 25
26
JMETC FY10 Accomplishments
JMETC Accomplishments
Supported 88 distinct customer test activities
Expanded network from 38 to 57 sites
ATIN and JTDL Networks transitioned to JMETC
Upgraded JMETC support applications and utilities to TENA R6
DIACAP Tiger Team report completed and recommendations being executed
Enhanced JMETC services and capabilities provided by leveraging InterTEC Services and Industry
Reuse Repository usability improvements
FY10 Example JMETC Customers
Joint Integrated Air amp Missile Defense Organization (JIAMDO)
Broad Area Communications Node (BACN) JUON
B1-B
Broad Area Maritime Surveillance System (BAMS)
Air-Ground Integrated Layer Exploration (AGILE)
Joint Interoperability Test Command (JITC)
Selected Benefits to the DoD
Integrated DT amp OT on a Joint Urgent Operational Need for the warfighter
Maximized usage of theater assets during limited maintenance windows
Improved Joint track information sharing to ensure interoperability of systems in theater operations
Coalition exchange and examination of real-time air picture data
Identification of Air Force Initiatives ready for warfighter transition
Investigated tactical UAS deployment in the National Airspace
Employment of Net-Enabled Weapons
JCAS immediate request amp end-to-end processes ldquoas-isrdquo characterization
Determined distributed system components were not ready for full live integration testing
Executed testing to support system-of-system interoperability certification
27
Customer Event Record Event Dates
Joint JIAMDO CorrelationDecorrelation Interoperability Test (CDIT)
Integration Events (Continuous) OCONUS October 2010
Navy UAS in NAS Runs for Record October 2010
Internal InterTEC Spiral 3 Systems Acceptance Test October - November 2010
Joint JITC Joint Interoperability Tests JIT 11-12345 (Continuous) October 2010 ndash September 2011
Air Force B1-B Fully Integrated Data Link Testing November 2010
Air Force AGILE Fire Phase III JEFX 2011 December 2010 - February 2011
Joint JTRS JPO -- JTRS Ground Mobile Radio January 2011
Navy Broad Area Maritime Surveillance (BAMS) (Continuous) January ndash September 2011
Joint Joint Track Manager Capability Demonstration (Continuous) January ndash September 2011
Joint JSF Initial MampS Interoperability (Continuous) February ndash March 2011
Air Force JSTARS Interoperability Test May 2011
Joint JS J8JIAMDO Joint Sensor Integration June ndash August 2011
FY11 JMETC Events
(More to Come)
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Global Hawk GATOR
Brigade Combat Team (BCT)
Modernization F-22 FY12 Testing Planned Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run
28
JMETC Connectivity
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca (2) JITC JTDL
Redstone (7) SMDC
RTC DTCC GMAN
SED Patriot THAAD FAAD SIV
Charleston (2)IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB SIMAF
As of 16 Mar 2011
0
10
20
30
40
50
60
70
FY06 FY07 FY08 FY09 FY10 FY11
JMETC Site Expansion
Sites
Dedicated trusted connectivity on SDREN (part of the GIG)
Encrypted for Secret ndash System High
DISA-registered IP address space
Active monitoring of network performance
Capable of supporting multiple simultaneous test events Bethpage NGC BAMS
Whiteman B-2
Eglin AFB (4)AOC DTF GWEF KHILS
Hurlburt Field C2DAC
Sites in Gulf Range
Functional Sites 61
New Sites Planned 8
Connection Points to Other Networks 5
Ft Greely CRTC
Site in Alaska
MHPCC
PMRF Bldg 105
Sites in Hawaii
Camp PendletonMCTSSA
China Lake (3)AV-8B FA-18IBAR
Edwards Ridley
Corona NSWC
El Segundo NGC B-2
Sites in SoCal
Point Loma (2)RLBTSSSC-PAC 59140 SSC-PAC HAIPE
Point Mugu (2)ITEC EA-6B
West Agg Rtr
Rancho Bernardo NGC BAMS
Pax River (5)ESTEL E2CD MCLACETEF SAIL ATR JMETC SYSCONEast Agg Rtr
Aberdeen ACCN
Langley (2) C-GIIF TDLITC
JFCOM JSIC
Wallops Island (2) SCSC SSDS
DISA Sky 7
Pentagon WARCAP
Dahlgren (2) CEDLIWSL
Dam Neck CDSA
Newport News NGC VASCIC
Sites in MD DC VA
Norfolk COMOPTEVFOR
McLean MITRE NCEL
Ft Huachuca JTRS
Bethpage NGC BCN
How an Air Force Customer Sees
Distributed Connectivity
Hardware-in-the-Loop (HWIL) Availability
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca JITC
Redstone (3) DTCC GMAN SED
Charleston (2) IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB
SIMAF
Bethpage NG BAMS
Whiteman B-2
MHPCC
PMRF Bldg 105
Sites in Hawaii
All linked by JMETC
29
Challenges Explained in Detail
30
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
2010 2030
Individual Systems WHAT do we test Mission-Level
amp Systems of Systems Capabilities
Events WHEN do we test Continuously
Decision-Makers For WHOM do we test Decision-Makers
amp Warfighters
Agile
Streamlined
Affordable
Continuous
Goal Distributed Integrated Testing and Training
The DoD TampE process must evolve to be
Test Resource Management Center
Future TampE Process
15
Aspects of the 2030 Common
Vision
bull Test amp Training bull Real-Time TTP SOP Development to improve SoS effectiveness and
suitability (limited)
bull ldquoBlack boxrdquo and ldquowhite boxrdquo testing done in parallel
including
bull System functionality
bull System effectiveness
bull System of Systems effectiveness via mission threads in an
operationally relevant environment
bull Fewer test runsconfigurations with more meaningful
collected data
bull Real-Time Data Analysis amp Automated Reporting
OBJECTIVE Near Real-Time Test-Fix-Test 16
Some Challenges to Achieving
2030 Vision
bull Authoritative Joint Mission threads
bull Mission level measures
bull Thorough but Streamlined Information Assurance
Procedures
bull Enterprise-level Multi-Level Security solutions
bull Embedded Encryption on acquisition systems
bull RF over the wire (network)
bull Near Real Time Test-Fix-Test
bull Data Management
bull Embedded amp Unobtrusive Instrumentation
bull World wide wireless
bull Extremely efficient usage of the frequency spectrum 17
Examples Where the Vision Is Starting
to Come Together
bull Joint Integrated Air amp Missile Defense Organization
(JIAMDO) Correlation Decorrelation Interoperability Test
(CDIT) is reducing analysis cycle from 3 months to less
than a month
bull Changes to test configuration between test runs (~30 minutes)
bull Overnight AdditionSubtraction of test runs depending on issues
discovered and data analysis results
bull Real-time impact to test configuration and number of test runs
bull Emphasis on evaluators and tools for detailed data analysis
bull Air-Ground Integrated Layer Exploration (AGILE) bull AGILE provides an LVC capability for acquisition amp testing
bull Analytical Rigor provides decision quality data for decision support
bull Efficient amp collaborative use of Joint analytical resources (in
addition to service capabilities)
bull Sustains and evolves collaborative environment for future use 18
Some TRMC Investments for
Achieving Common Vision
bull Central TampE Investment Program (CTEIP)
bull Test and Training Enabling Architecture (TENA)
bull Multi-Level Security-Joint Coalition Network Environment (MLS-
JCNE)
bull InterTEC
bull Result JITC-certified instrumentation usable in all aspects of TampE
bull Common Range Integrated Instrumentation System (CRIIS)
bull TampE SampT Program
bull TENA in a Resource Constrained Environment (TRCE)
bull NST Evaluation Capability Model (NECM)
bull JMETC Program
bull DoD corporate approach for distributed testing 19
What Can You Do to Be Ready for
2030
bull Maintain Information Assurance agreements for all facility
systems
bull Streamline data collection amp analysis at your facility
bull Get comfortable with DoDAF views
bull Look across the RDTampE the community to find common
problems that can be addressed with similar solutions
bull TENA-enable your instrumentation systems and test
command amp control
bull Get involved share your requirements with your Service
representatives and in TRMC users meetings
20
JMETC Users Group
bull Purpose is to focus on technical requirements and solutions relevant to current and future Distributed Testing needs
bull Technical and Management level representatives identify core infrastructure requirements and most importantly resolve issues
bull Quarterly meetings of 250-300 JMETC customers acquisition programs test events ranges LVC sites tools and network providers
21
Last JMETC Users Group Meeting bull June 2011 bull Location Mid-East Coast bull Tracks
bull User Requirements bull Information Assurance
Security bull Data Management bull InterTEC (Current amp Planned) bull Networking
bull An established forum for the Distributed Test Community to
o Identify core infrastructure requirements and use cases
o Identify investigate amp resolve issues
o Identify opportunities to collaborate
o Discuss available solutions tools and techniques
o Share lessons learned
Summary
bull We are becoming more efficient in how we execute TampE
bull We must get the biggest and best ldquobang for the buckrdquo that we can
bull TRMC and Services continue TampE investments to enable realization of the 2030 Vision
bull Align TampE (OSD and Services) investments to mitigate test risk areas
bull The Future of TampE will include Distributed Integrated Testing and Training and will enable near real time Test-Fix-Test
22
JMETC Program Points of Contact
23
JMETC Program Manager Chip Ferguson chipfergusonosdmil 703-601-5274
JMETC Principal Deputy PM Bruce Bailey brucebaileyosdmil 703-601-5208
JMETC Lead Operations Planning Marty Arnwine martemasarnwineosdmil 703-601-5215
JMETC Senior Technical Advisor George Rumford georgerumfordosdmil
703-601-5233
JMETC Lead Systems Engineer Ryan Norman ryannormanosdmil
703-601-5277
JMETC Lead Network Engineer Arjuna ldquoAJrdquo Pathmanathan ArjunaPathmanathanosdmil
703-601-5214
JMETC Website wwwjmetcorg
Backup Slides
25
FY10 JMETC Events
Customer Event Record Event Dates
Air Force JEFX 10-1Spirit Ice (B2 Data Link Test) October ndash November 2009
Navy BAMS LVC DE October ndash December 2009
Air Force Battlefield Airborne Comm Node (BACN) JUON
(DTOT)
November 2009 to September 2010
Air Force Agile Fire 10-2 January 2010
Air Force JEFX 10-23 FebruaryApril 2010
Joint Joint Surface Warfare JCTD February to September 2010
JS J8JIAMDO Joint Sensor Integration April to September 2010
Air Force B1-B Fully Integrated Data Link Testing April 2010
JFCOM J8489 (TEST WEEK) JCAS Distributed Test June 2010
JIAMDO (Navy Lead) CorrelationDecorrelation Interoperability Test
(CDIT) Integration Events (Continuous)
July to September 2010
Army (Lead) UAS in the National Airspace July to September 2010
Air Force Agile Fire 10-3 August 2010
Joint JITC Joint Interoperability Tests JIT 10-3 amp 11-1
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Joint Strike Fighter (JSF) JIAMDOJoint Track Manager
Brigade Combat Team (BCT)
Modernization
Multi-Function Adv Data Link (MADL) Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run 25
26
JMETC FY10 Accomplishments
JMETC Accomplishments
Supported 88 distinct customer test activities
Expanded network from 38 to 57 sites
ATIN and JTDL Networks transitioned to JMETC
Upgraded JMETC support applications and utilities to TENA R6
DIACAP Tiger Team report completed and recommendations being executed
Enhanced JMETC services and capabilities provided by leveraging InterTEC Services and Industry
Reuse Repository usability improvements
FY10 Example JMETC Customers
Joint Integrated Air amp Missile Defense Organization (JIAMDO)
Broad Area Communications Node (BACN) JUON
B1-B
Broad Area Maritime Surveillance System (BAMS)
Air-Ground Integrated Layer Exploration (AGILE)
Joint Interoperability Test Command (JITC)
Selected Benefits to the DoD
Integrated DT amp OT on a Joint Urgent Operational Need for the warfighter
Maximized usage of theater assets during limited maintenance windows
Improved Joint track information sharing to ensure interoperability of systems in theater operations
Coalition exchange and examination of real-time air picture data
Identification of Air Force Initiatives ready for warfighter transition
Investigated tactical UAS deployment in the National Airspace
Employment of Net-Enabled Weapons
JCAS immediate request amp end-to-end processes ldquoas-isrdquo characterization
Determined distributed system components were not ready for full live integration testing
Executed testing to support system-of-system interoperability certification
27
Customer Event Record Event Dates
Joint JIAMDO CorrelationDecorrelation Interoperability Test (CDIT)
Integration Events (Continuous) OCONUS October 2010
Navy UAS in NAS Runs for Record October 2010
Internal InterTEC Spiral 3 Systems Acceptance Test October - November 2010
Joint JITC Joint Interoperability Tests JIT 11-12345 (Continuous) October 2010 ndash September 2011
Air Force B1-B Fully Integrated Data Link Testing November 2010
Air Force AGILE Fire Phase III JEFX 2011 December 2010 - February 2011
Joint JTRS JPO -- JTRS Ground Mobile Radio January 2011
Navy Broad Area Maritime Surveillance (BAMS) (Continuous) January ndash September 2011
Joint Joint Track Manager Capability Demonstration (Continuous) January ndash September 2011
Joint JSF Initial MampS Interoperability (Continuous) February ndash March 2011
Air Force JSTARS Interoperability Test May 2011
Joint JS J8JIAMDO Joint Sensor Integration June ndash August 2011
FY11 JMETC Events
(More to Come)
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Global Hawk GATOR
Brigade Combat Team (BCT)
Modernization F-22 FY12 Testing Planned Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run
28
JMETC Connectivity
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca (2) JITC JTDL
Redstone (7) SMDC
RTC DTCC GMAN
SED Patriot THAAD FAAD SIV
Charleston (2)IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB SIMAF
As of 16 Mar 2011
0
10
20
30
40
50
60
70
FY06 FY07 FY08 FY09 FY10 FY11
JMETC Site Expansion
Sites
Dedicated trusted connectivity on SDREN (part of the GIG)
Encrypted for Secret ndash System High
DISA-registered IP address space
Active monitoring of network performance
Capable of supporting multiple simultaneous test events Bethpage NGC BAMS
Whiteman B-2
Eglin AFB (4)AOC DTF GWEF KHILS
Hurlburt Field C2DAC
Sites in Gulf Range
Functional Sites 61
New Sites Planned 8
Connection Points to Other Networks 5
Ft Greely CRTC
Site in Alaska
MHPCC
PMRF Bldg 105
Sites in Hawaii
Camp PendletonMCTSSA
China Lake (3)AV-8B FA-18IBAR
Edwards Ridley
Corona NSWC
El Segundo NGC B-2
Sites in SoCal
Point Loma (2)RLBTSSSC-PAC 59140 SSC-PAC HAIPE
Point Mugu (2)ITEC EA-6B
West Agg Rtr
Rancho Bernardo NGC BAMS
Pax River (5)ESTEL E2CD MCLACETEF SAIL ATR JMETC SYSCONEast Agg Rtr
Aberdeen ACCN
Langley (2) C-GIIF TDLITC
JFCOM JSIC
Wallops Island (2) SCSC SSDS
DISA Sky 7
Pentagon WARCAP
Dahlgren (2) CEDLIWSL
Dam Neck CDSA
Newport News NGC VASCIC
Sites in MD DC VA
Norfolk COMOPTEVFOR
McLean MITRE NCEL
Ft Huachuca JTRS
Bethpage NGC BCN
How an Air Force Customer Sees
Distributed Connectivity
Hardware-in-the-Loop (HWIL) Availability
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca JITC
Redstone (3) DTCC GMAN SED
Charleston (2) IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB
SIMAF
Bethpage NG BAMS
Whiteman B-2
MHPCC
PMRF Bldg 105
Sites in Hawaii
All linked by JMETC
29
Challenges Explained in Detail
30
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
Aspects of the 2030 Common
Vision
bull Test amp Training bull Real-Time TTP SOP Development to improve SoS effectiveness and
suitability (limited)
bull ldquoBlack boxrdquo and ldquowhite boxrdquo testing done in parallel
including
bull System functionality
bull System effectiveness
bull System of Systems effectiveness via mission threads in an
operationally relevant environment
bull Fewer test runsconfigurations with more meaningful
collected data
bull Real-Time Data Analysis amp Automated Reporting
OBJECTIVE Near Real-Time Test-Fix-Test 16
Some Challenges to Achieving
2030 Vision
bull Authoritative Joint Mission threads
bull Mission level measures
bull Thorough but Streamlined Information Assurance
Procedures
bull Enterprise-level Multi-Level Security solutions
bull Embedded Encryption on acquisition systems
bull RF over the wire (network)
bull Near Real Time Test-Fix-Test
bull Data Management
bull Embedded amp Unobtrusive Instrumentation
bull World wide wireless
bull Extremely efficient usage of the frequency spectrum 17
Examples Where the Vision Is Starting
to Come Together
bull Joint Integrated Air amp Missile Defense Organization
(JIAMDO) Correlation Decorrelation Interoperability Test
(CDIT) is reducing analysis cycle from 3 months to less
than a month
bull Changes to test configuration between test runs (~30 minutes)
bull Overnight AdditionSubtraction of test runs depending on issues
discovered and data analysis results
bull Real-time impact to test configuration and number of test runs
bull Emphasis on evaluators and tools for detailed data analysis
bull Air-Ground Integrated Layer Exploration (AGILE) bull AGILE provides an LVC capability for acquisition amp testing
bull Analytical Rigor provides decision quality data for decision support
bull Efficient amp collaborative use of Joint analytical resources (in
addition to service capabilities)
bull Sustains and evolves collaborative environment for future use 18
Some TRMC Investments for
Achieving Common Vision
bull Central TampE Investment Program (CTEIP)
bull Test and Training Enabling Architecture (TENA)
bull Multi-Level Security-Joint Coalition Network Environment (MLS-
JCNE)
bull InterTEC
bull Result JITC-certified instrumentation usable in all aspects of TampE
bull Common Range Integrated Instrumentation System (CRIIS)
bull TampE SampT Program
bull TENA in a Resource Constrained Environment (TRCE)
bull NST Evaluation Capability Model (NECM)
bull JMETC Program
bull DoD corporate approach for distributed testing 19
What Can You Do to Be Ready for
2030
bull Maintain Information Assurance agreements for all facility
systems
bull Streamline data collection amp analysis at your facility
bull Get comfortable with DoDAF views
bull Look across the RDTampE the community to find common
problems that can be addressed with similar solutions
bull TENA-enable your instrumentation systems and test
command amp control
bull Get involved share your requirements with your Service
representatives and in TRMC users meetings
20
JMETC Users Group
bull Purpose is to focus on technical requirements and solutions relevant to current and future Distributed Testing needs
bull Technical and Management level representatives identify core infrastructure requirements and most importantly resolve issues
bull Quarterly meetings of 250-300 JMETC customers acquisition programs test events ranges LVC sites tools and network providers
21
Last JMETC Users Group Meeting bull June 2011 bull Location Mid-East Coast bull Tracks
bull User Requirements bull Information Assurance
Security bull Data Management bull InterTEC (Current amp Planned) bull Networking
bull An established forum for the Distributed Test Community to
o Identify core infrastructure requirements and use cases
o Identify investigate amp resolve issues
o Identify opportunities to collaborate
o Discuss available solutions tools and techniques
o Share lessons learned
Summary
bull We are becoming more efficient in how we execute TampE
bull We must get the biggest and best ldquobang for the buckrdquo that we can
bull TRMC and Services continue TampE investments to enable realization of the 2030 Vision
bull Align TampE (OSD and Services) investments to mitigate test risk areas
bull The Future of TampE will include Distributed Integrated Testing and Training and will enable near real time Test-Fix-Test
22
JMETC Program Points of Contact
23
JMETC Program Manager Chip Ferguson chipfergusonosdmil 703-601-5274
JMETC Principal Deputy PM Bruce Bailey brucebaileyosdmil 703-601-5208
JMETC Lead Operations Planning Marty Arnwine martemasarnwineosdmil 703-601-5215
JMETC Senior Technical Advisor George Rumford georgerumfordosdmil
703-601-5233
JMETC Lead Systems Engineer Ryan Norman ryannormanosdmil
703-601-5277
JMETC Lead Network Engineer Arjuna ldquoAJrdquo Pathmanathan ArjunaPathmanathanosdmil
703-601-5214
JMETC Website wwwjmetcorg
Backup Slides
25
FY10 JMETC Events
Customer Event Record Event Dates
Air Force JEFX 10-1Spirit Ice (B2 Data Link Test) October ndash November 2009
Navy BAMS LVC DE October ndash December 2009
Air Force Battlefield Airborne Comm Node (BACN) JUON
(DTOT)
November 2009 to September 2010
Air Force Agile Fire 10-2 January 2010
Air Force JEFX 10-23 FebruaryApril 2010
Joint Joint Surface Warfare JCTD February to September 2010
JS J8JIAMDO Joint Sensor Integration April to September 2010
Air Force B1-B Fully Integrated Data Link Testing April 2010
JFCOM J8489 (TEST WEEK) JCAS Distributed Test June 2010
JIAMDO (Navy Lead) CorrelationDecorrelation Interoperability Test
(CDIT) Integration Events (Continuous)
July to September 2010
Army (Lead) UAS in the National Airspace July to September 2010
Air Force Agile Fire 10-3 August 2010
Joint JITC Joint Interoperability Tests JIT 10-3 amp 11-1
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Joint Strike Fighter (JSF) JIAMDOJoint Track Manager
Brigade Combat Team (BCT)
Modernization
Multi-Function Adv Data Link (MADL) Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run 25
26
JMETC FY10 Accomplishments
JMETC Accomplishments
Supported 88 distinct customer test activities
Expanded network from 38 to 57 sites
ATIN and JTDL Networks transitioned to JMETC
Upgraded JMETC support applications and utilities to TENA R6
DIACAP Tiger Team report completed and recommendations being executed
Enhanced JMETC services and capabilities provided by leveraging InterTEC Services and Industry
Reuse Repository usability improvements
FY10 Example JMETC Customers
Joint Integrated Air amp Missile Defense Organization (JIAMDO)
Broad Area Communications Node (BACN) JUON
B1-B
Broad Area Maritime Surveillance System (BAMS)
Air-Ground Integrated Layer Exploration (AGILE)
Joint Interoperability Test Command (JITC)
Selected Benefits to the DoD
Integrated DT amp OT on a Joint Urgent Operational Need for the warfighter
Maximized usage of theater assets during limited maintenance windows
Improved Joint track information sharing to ensure interoperability of systems in theater operations
Coalition exchange and examination of real-time air picture data
Identification of Air Force Initiatives ready for warfighter transition
Investigated tactical UAS deployment in the National Airspace
Employment of Net-Enabled Weapons
JCAS immediate request amp end-to-end processes ldquoas-isrdquo characterization
Determined distributed system components were not ready for full live integration testing
Executed testing to support system-of-system interoperability certification
27
Customer Event Record Event Dates
Joint JIAMDO CorrelationDecorrelation Interoperability Test (CDIT)
Integration Events (Continuous) OCONUS October 2010
Navy UAS in NAS Runs for Record October 2010
Internal InterTEC Spiral 3 Systems Acceptance Test October - November 2010
Joint JITC Joint Interoperability Tests JIT 11-12345 (Continuous) October 2010 ndash September 2011
Air Force B1-B Fully Integrated Data Link Testing November 2010
Air Force AGILE Fire Phase III JEFX 2011 December 2010 - February 2011
Joint JTRS JPO -- JTRS Ground Mobile Radio January 2011
Navy Broad Area Maritime Surveillance (BAMS) (Continuous) January ndash September 2011
Joint Joint Track Manager Capability Demonstration (Continuous) January ndash September 2011
Joint JSF Initial MampS Interoperability (Continuous) February ndash March 2011
Air Force JSTARS Interoperability Test May 2011
Joint JS J8JIAMDO Joint Sensor Integration June ndash August 2011
FY11 JMETC Events
(More to Come)
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Global Hawk GATOR
Brigade Combat Team (BCT)
Modernization F-22 FY12 Testing Planned Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run
28
JMETC Connectivity
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca (2) JITC JTDL
Redstone (7) SMDC
RTC DTCC GMAN
SED Patriot THAAD FAAD SIV
Charleston (2)IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB SIMAF
As of 16 Mar 2011
0
10
20
30
40
50
60
70
FY06 FY07 FY08 FY09 FY10 FY11
JMETC Site Expansion
Sites
Dedicated trusted connectivity on SDREN (part of the GIG)
Encrypted for Secret ndash System High
DISA-registered IP address space
Active monitoring of network performance
Capable of supporting multiple simultaneous test events Bethpage NGC BAMS
Whiteman B-2
Eglin AFB (4)AOC DTF GWEF KHILS
Hurlburt Field C2DAC
Sites in Gulf Range
Functional Sites 61
New Sites Planned 8
Connection Points to Other Networks 5
Ft Greely CRTC
Site in Alaska
MHPCC
PMRF Bldg 105
Sites in Hawaii
Camp PendletonMCTSSA
China Lake (3)AV-8B FA-18IBAR
Edwards Ridley
Corona NSWC
El Segundo NGC B-2
Sites in SoCal
Point Loma (2)RLBTSSSC-PAC 59140 SSC-PAC HAIPE
Point Mugu (2)ITEC EA-6B
West Agg Rtr
Rancho Bernardo NGC BAMS
Pax River (5)ESTEL E2CD MCLACETEF SAIL ATR JMETC SYSCONEast Agg Rtr
Aberdeen ACCN
Langley (2) C-GIIF TDLITC
JFCOM JSIC
Wallops Island (2) SCSC SSDS
DISA Sky 7
Pentagon WARCAP
Dahlgren (2) CEDLIWSL
Dam Neck CDSA
Newport News NGC VASCIC
Sites in MD DC VA
Norfolk COMOPTEVFOR
McLean MITRE NCEL
Ft Huachuca JTRS
Bethpage NGC BCN
How an Air Force Customer Sees
Distributed Connectivity
Hardware-in-the-Loop (HWIL) Availability
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca JITC
Redstone (3) DTCC GMAN SED
Charleston (2) IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB
SIMAF
Bethpage NG BAMS
Whiteman B-2
MHPCC
PMRF Bldg 105
Sites in Hawaii
All linked by JMETC
29
Challenges Explained in Detail
30
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
Some Challenges to Achieving
2030 Vision
bull Authoritative Joint Mission threads
bull Mission level measures
bull Thorough but Streamlined Information Assurance
Procedures
bull Enterprise-level Multi-Level Security solutions
bull Embedded Encryption on acquisition systems
bull RF over the wire (network)
bull Near Real Time Test-Fix-Test
bull Data Management
bull Embedded amp Unobtrusive Instrumentation
bull World wide wireless
bull Extremely efficient usage of the frequency spectrum 17
Examples Where the Vision Is Starting
to Come Together
bull Joint Integrated Air amp Missile Defense Organization
(JIAMDO) Correlation Decorrelation Interoperability Test
(CDIT) is reducing analysis cycle from 3 months to less
than a month
bull Changes to test configuration between test runs (~30 minutes)
bull Overnight AdditionSubtraction of test runs depending on issues
discovered and data analysis results
bull Real-time impact to test configuration and number of test runs
bull Emphasis on evaluators and tools for detailed data analysis
bull Air-Ground Integrated Layer Exploration (AGILE) bull AGILE provides an LVC capability for acquisition amp testing
bull Analytical Rigor provides decision quality data for decision support
bull Efficient amp collaborative use of Joint analytical resources (in
addition to service capabilities)
bull Sustains and evolves collaborative environment for future use 18
Some TRMC Investments for
Achieving Common Vision
bull Central TampE Investment Program (CTEIP)
bull Test and Training Enabling Architecture (TENA)
bull Multi-Level Security-Joint Coalition Network Environment (MLS-
JCNE)
bull InterTEC
bull Result JITC-certified instrumentation usable in all aspects of TampE
bull Common Range Integrated Instrumentation System (CRIIS)
bull TampE SampT Program
bull TENA in a Resource Constrained Environment (TRCE)
bull NST Evaluation Capability Model (NECM)
bull JMETC Program
bull DoD corporate approach for distributed testing 19
What Can You Do to Be Ready for
2030
bull Maintain Information Assurance agreements for all facility
systems
bull Streamline data collection amp analysis at your facility
bull Get comfortable with DoDAF views
bull Look across the RDTampE the community to find common
problems that can be addressed with similar solutions
bull TENA-enable your instrumentation systems and test
command amp control
bull Get involved share your requirements with your Service
representatives and in TRMC users meetings
20
JMETC Users Group
bull Purpose is to focus on technical requirements and solutions relevant to current and future Distributed Testing needs
bull Technical and Management level representatives identify core infrastructure requirements and most importantly resolve issues
bull Quarterly meetings of 250-300 JMETC customers acquisition programs test events ranges LVC sites tools and network providers
21
Last JMETC Users Group Meeting bull June 2011 bull Location Mid-East Coast bull Tracks
bull User Requirements bull Information Assurance
Security bull Data Management bull InterTEC (Current amp Planned) bull Networking
bull An established forum for the Distributed Test Community to
o Identify core infrastructure requirements and use cases
o Identify investigate amp resolve issues
o Identify opportunities to collaborate
o Discuss available solutions tools and techniques
o Share lessons learned
Summary
bull We are becoming more efficient in how we execute TampE
bull We must get the biggest and best ldquobang for the buckrdquo that we can
bull TRMC and Services continue TampE investments to enable realization of the 2030 Vision
bull Align TampE (OSD and Services) investments to mitigate test risk areas
bull The Future of TampE will include Distributed Integrated Testing and Training and will enable near real time Test-Fix-Test
22
JMETC Program Points of Contact
23
JMETC Program Manager Chip Ferguson chipfergusonosdmil 703-601-5274
JMETC Principal Deputy PM Bruce Bailey brucebaileyosdmil 703-601-5208
JMETC Lead Operations Planning Marty Arnwine martemasarnwineosdmil 703-601-5215
JMETC Senior Technical Advisor George Rumford georgerumfordosdmil
703-601-5233
JMETC Lead Systems Engineer Ryan Norman ryannormanosdmil
703-601-5277
JMETC Lead Network Engineer Arjuna ldquoAJrdquo Pathmanathan ArjunaPathmanathanosdmil
703-601-5214
JMETC Website wwwjmetcorg
Backup Slides
25
FY10 JMETC Events
Customer Event Record Event Dates
Air Force JEFX 10-1Spirit Ice (B2 Data Link Test) October ndash November 2009
Navy BAMS LVC DE October ndash December 2009
Air Force Battlefield Airborne Comm Node (BACN) JUON
(DTOT)
November 2009 to September 2010
Air Force Agile Fire 10-2 January 2010
Air Force JEFX 10-23 FebruaryApril 2010
Joint Joint Surface Warfare JCTD February to September 2010
JS J8JIAMDO Joint Sensor Integration April to September 2010
Air Force B1-B Fully Integrated Data Link Testing April 2010
JFCOM J8489 (TEST WEEK) JCAS Distributed Test June 2010
JIAMDO (Navy Lead) CorrelationDecorrelation Interoperability Test
(CDIT) Integration Events (Continuous)
July to September 2010
Army (Lead) UAS in the National Airspace July to September 2010
Air Force Agile Fire 10-3 August 2010
Joint JITC Joint Interoperability Tests JIT 10-3 amp 11-1
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Joint Strike Fighter (JSF) JIAMDOJoint Track Manager
Brigade Combat Team (BCT)
Modernization
Multi-Function Adv Data Link (MADL) Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run 25
26
JMETC FY10 Accomplishments
JMETC Accomplishments
Supported 88 distinct customer test activities
Expanded network from 38 to 57 sites
ATIN and JTDL Networks transitioned to JMETC
Upgraded JMETC support applications and utilities to TENA R6
DIACAP Tiger Team report completed and recommendations being executed
Enhanced JMETC services and capabilities provided by leveraging InterTEC Services and Industry
Reuse Repository usability improvements
FY10 Example JMETC Customers
Joint Integrated Air amp Missile Defense Organization (JIAMDO)
Broad Area Communications Node (BACN) JUON
B1-B
Broad Area Maritime Surveillance System (BAMS)
Air-Ground Integrated Layer Exploration (AGILE)
Joint Interoperability Test Command (JITC)
Selected Benefits to the DoD
Integrated DT amp OT on a Joint Urgent Operational Need for the warfighter
Maximized usage of theater assets during limited maintenance windows
Improved Joint track information sharing to ensure interoperability of systems in theater operations
Coalition exchange and examination of real-time air picture data
Identification of Air Force Initiatives ready for warfighter transition
Investigated tactical UAS deployment in the National Airspace
Employment of Net-Enabled Weapons
JCAS immediate request amp end-to-end processes ldquoas-isrdquo characterization
Determined distributed system components were not ready for full live integration testing
Executed testing to support system-of-system interoperability certification
27
Customer Event Record Event Dates
Joint JIAMDO CorrelationDecorrelation Interoperability Test (CDIT)
Integration Events (Continuous) OCONUS October 2010
Navy UAS in NAS Runs for Record October 2010
Internal InterTEC Spiral 3 Systems Acceptance Test October - November 2010
Joint JITC Joint Interoperability Tests JIT 11-12345 (Continuous) October 2010 ndash September 2011
Air Force B1-B Fully Integrated Data Link Testing November 2010
Air Force AGILE Fire Phase III JEFX 2011 December 2010 - February 2011
Joint JTRS JPO -- JTRS Ground Mobile Radio January 2011
Navy Broad Area Maritime Surveillance (BAMS) (Continuous) January ndash September 2011
Joint Joint Track Manager Capability Demonstration (Continuous) January ndash September 2011
Joint JSF Initial MampS Interoperability (Continuous) February ndash March 2011
Air Force JSTARS Interoperability Test May 2011
Joint JS J8JIAMDO Joint Sensor Integration June ndash August 2011
FY11 JMETC Events
(More to Come)
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Global Hawk GATOR
Brigade Combat Team (BCT)
Modernization F-22 FY12 Testing Planned Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run
28
JMETC Connectivity
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca (2) JITC JTDL
Redstone (7) SMDC
RTC DTCC GMAN
SED Patriot THAAD FAAD SIV
Charleston (2)IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB SIMAF
As of 16 Mar 2011
0
10
20
30
40
50
60
70
FY06 FY07 FY08 FY09 FY10 FY11
JMETC Site Expansion
Sites
Dedicated trusted connectivity on SDREN (part of the GIG)
Encrypted for Secret ndash System High
DISA-registered IP address space
Active monitoring of network performance
Capable of supporting multiple simultaneous test events Bethpage NGC BAMS
Whiteman B-2
Eglin AFB (4)AOC DTF GWEF KHILS
Hurlburt Field C2DAC
Sites in Gulf Range
Functional Sites 61
New Sites Planned 8
Connection Points to Other Networks 5
Ft Greely CRTC
Site in Alaska
MHPCC
PMRF Bldg 105
Sites in Hawaii
Camp PendletonMCTSSA
China Lake (3)AV-8B FA-18IBAR
Edwards Ridley
Corona NSWC
El Segundo NGC B-2
Sites in SoCal
Point Loma (2)RLBTSSSC-PAC 59140 SSC-PAC HAIPE
Point Mugu (2)ITEC EA-6B
West Agg Rtr
Rancho Bernardo NGC BAMS
Pax River (5)ESTEL E2CD MCLACETEF SAIL ATR JMETC SYSCONEast Agg Rtr
Aberdeen ACCN
Langley (2) C-GIIF TDLITC
JFCOM JSIC
Wallops Island (2) SCSC SSDS
DISA Sky 7
Pentagon WARCAP
Dahlgren (2) CEDLIWSL
Dam Neck CDSA
Newport News NGC VASCIC
Sites in MD DC VA
Norfolk COMOPTEVFOR
McLean MITRE NCEL
Ft Huachuca JTRS
Bethpage NGC BCN
How an Air Force Customer Sees
Distributed Connectivity
Hardware-in-the-Loop (HWIL) Availability
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca JITC
Redstone (3) DTCC GMAN SED
Charleston (2) IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB
SIMAF
Bethpage NG BAMS
Whiteman B-2
MHPCC
PMRF Bldg 105
Sites in Hawaii
All linked by JMETC
29
Challenges Explained in Detail
30
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
Examples Where the Vision Is Starting
to Come Together
bull Joint Integrated Air amp Missile Defense Organization
(JIAMDO) Correlation Decorrelation Interoperability Test
(CDIT) is reducing analysis cycle from 3 months to less
than a month
bull Changes to test configuration between test runs (~30 minutes)
bull Overnight AdditionSubtraction of test runs depending on issues
discovered and data analysis results
bull Real-time impact to test configuration and number of test runs
bull Emphasis on evaluators and tools for detailed data analysis
bull Air-Ground Integrated Layer Exploration (AGILE) bull AGILE provides an LVC capability for acquisition amp testing
bull Analytical Rigor provides decision quality data for decision support
bull Efficient amp collaborative use of Joint analytical resources (in
addition to service capabilities)
bull Sustains and evolves collaborative environment for future use 18
Some TRMC Investments for
Achieving Common Vision
bull Central TampE Investment Program (CTEIP)
bull Test and Training Enabling Architecture (TENA)
bull Multi-Level Security-Joint Coalition Network Environment (MLS-
JCNE)
bull InterTEC
bull Result JITC-certified instrumentation usable in all aspects of TampE
bull Common Range Integrated Instrumentation System (CRIIS)
bull TampE SampT Program
bull TENA in a Resource Constrained Environment (TRCE)
bull NST Evaluation Capability Model (NECM)
bull JMETC Program
bull DoD corporate approach for distributed testing 19
What Can You Do to Be Ready for
2030
bull Maintain Information Assurance agreements for all facility
systems
bull Streamline data collection amp analysis at your facility
bull Get comfortable with DoDAF views
bull Look across the RDTampE the community to find common
problems that can be addressed with similar solutions
bull TENA-enable your instrumentation systems and test
command amp control
bull Get involved share your requirements with your Service
representatives and in TRMC users meetings
20
JMETC Users Group
bull Purpose is to focus on technical requirements and solutions relevant to current and future Distributed Testing needs
bull Technical and Management level representatives identify core infrastructure requirements and most importantly resolve issues
bull Quarterly meetings of 250-300 JMETC customers acquisition programs test events ranges LVC sites tools and network providers
21
Last JMETC Users Group Meeting bull June 2011 bull Location Mid-East Coast bull Tracks
bull User Requirements bull Information Assurance
Security bull Data Management bull InterTEC (Current amp Planned) bull Networking
bull An established forum for the Distributed Test Community to
o Identify core infrastructure requirements and use cases
o Identify investigate amp resolve issues
o Identify opportunities to collaborate
o Discuss available solutions tools and techniques
o Share lessons learned
Summary
bull We are becoming more efficient in how we execute TampE
bull We must get the biggest and best ldquobang for the buckrdquo that we can
bull TRMC and Services continue TampE investments to enable realization of the 2030 Vision
bull Align TampE (OSD and Services) investments to mitigate test risk areas
bull The Future of TampE will include Distributed Integrated Testing and Training and will enable near real time Test-Fix-Test
22
JMETC Program Points of Contact
23
JMETC Program Manager Chip Ferguson chipfergusonosdmil 703-601-5274
JMETC Principal Deputy PM Bruce Bailey brucebaileyosdmil 703-601-5208
JMETC Lead Operations Planning Marty Arnwine martemasarnwineosdmil 703-601-5215
JMETC Senior Technical Advisor George Rumford georgerumfordosdmil
703-601-5233
JMETC Lead Systems Engineer Ryan Norman ryannormanosdmil
703-601-5277
JMETC Lead Network Engineer Arjuna ldquoAJrdquo Pathmanathan ArjunaPathmanathanosdmil
703-601-5214
JMETC Website wwwjmetcorg
Backup Slides
25
FY10 JMETC Events
Customer Event Record Event Dates
Air Force JEFX 10-1Spirit Ice (B2 Data Link Test) October ndash November 2009
Navy BAMS LVC DE October ndash December 2009
Air Force Battlefield Airborne Comm Node (BACN) JUON
(DTOT)
November 2009 to September 2010
Air Force Agile Fire 10-2 January 2010
Air Force JEFX 10-23 FebruaryApril 2010
Joint Joint Surface Warfare JCTD February to September 2010
JS J8JIAMDO Joint Sensor Integration April to September 2010
Air Force B1-B Fully Integrated Data Link Testing April 2010
JFCOM J8489 (TEST WEEK) JCAS Distributed Test June 2010
JIAMDO (Navy Lead) CorrelationDecorrelation Interoperability Test
(CDIT) Integration Events (Continuous)
July to September 2010
Army (Lead) UAS in the National Airspace July to September 2010
Air Force Agile Fire 10-3 August 2010
Joint JITC Joint Interoperability Tests JIT 10-3 amp 11-1
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Joint Strike Fighter (JSF) JIAMDOJoint Track Manager
Brigade Combat Team (BCT)
Modernization
Multi-Function Adv Data Link (MADL) Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run 25
26
JMETC FY10 Accomplishments
JMETC Accomplishments
Supported 88 distinct customer test activities
Expanded network from 38 to 57 sites
ATIN and JTDL Networks transitioned to JMETC
Upgraded JMETC support applications and utilities to TENA R6
DIACAP Tiger Team report completed and recommendations being executed
Enhanced JMETC services and capabilities provided by leveraging InterTEC Services and Industry
Reuse Repository usability improvements
FY10 Example JMETC Customers
Joint Integrated Air amp Missile Defense Organization (JIAMDO)
Broad Area Communications Node (BACN) JUON
B1-B
Broad Area Maritime Surveillance System (BAMS)
Air-Ground Integrated Layer Exploration (AGILE)
Joint Interoperability Test Command (JITC)
Selected Benefits to the DoD
Integrated DT amp OT on a Joint Urgent Operational Need for the warfighter
Maximized usage of theater assets during limited maintenance windows
Improved Joint track information sharing to ensure interoperability of systems in theater operations
Coalition exchange and examination of real-time air picture data
Identification of Air Force Initiatives ready for warfighter transition
Investigated tactical UAS deployment in the National Airspace
Employment of Net-Enabled Weapons
JCAS immediate request amp end-to-end processes ldquoas-isrdquo characterization
Determined distributed system components were not ready for full live integration testing
Executed testing to support system-of-system interoperability certification
27
Customer Event Record Event Dates
Joint JIAMDO CorrelationDecorrelation Interoperability Test (CDIT)
Integration Events (Continuous) OCONUS October 2010
Navy UAS in NAS Runs for Record October 2010
Internal InterTEC Spiral 3 Systems Acceptance Test October - November 2010
Joint JITC Joint Interoperability Tests JIT 11-12345 (Continuous) October 2010 ndash September 2011
Air Force B1-B Fully Integrated Data Link Testing November 2010
Air Force AGILE Fire Phase III JEFX 2011 December 2010 - February 2011
Joint JTRS JPO -- JTRS Ground Mobile Radio January 2011
Navy Broad Area Maritime Surveillance (BAMS) (Continuous) January ndash September 2011
Joint Joint Track Manager Capability Demonstration (Continuous) January ndash September 2011
Joint JSF Initial MampS Interoperability (Continuous) February ndash March 2011
Air Force JSTARS Interoperability Test May 2011
Joint JS J8JIAMDO Joint Sensor Integration June ndash August 2011
FY11 JMETC Events
(More to Come)
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Global Hawk GATOR
Brigade Combat Team (BCT)
Modernization F-22 FY12 Testing Planned Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run
28
JMETC Connectivity
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca (2) JITC JTDL
Redstone (7) SMDC
RTC DTCC GMAN
SED Patriot THAAD FAAD SIV
Charleston (2)IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB SIMAF
As of 16 Mar 2011
0
10
20
30
40
50
60
70
FY06 FY07 FY08 FY09 FY10 FY11
JMETC Site Expansion
Sites
Dedicated trusted connectivity on SDREN (part of the GIG)
Encrypted for Secret ndash System High
DISA-registered IP address space
Active monitoring of network performance
Capable of supporting multiple simultaneous test events Bethpage NGC BAMS
Whiteman B-2
Eglin AFB (4)AOC DTF GWEF KHILS
Hurlburt Field C2DAC
Sites in Gulf Range
Functional Sites 61
New Sites Planned 8
Connection Points to Other Networks 5
Ft Greely CRTC
Site in Alaska
MHPCC
PMRF Bldg 105
Sites in Hawaii
Camp PendletonMCTSSA
China Lake (3)AV-8B FA-18IBAR
Edwards Ridley
Corona NSWC
El Segundo NGC B-2
Sites in SoCal
Point Loma (2)RLBTSSSC-PAC 59140 SSC-PAC HAIPE
Point Mugu (2)ITEC EA-6B
West Agg Rtr
Rancho Bernardo NGC BAMS
Pax River (5)ESTEL E2CD MCLACETEF SAIL ATR JMETC SYSCONEast Agg Rtr
Aberdeen ACCN
Langley (2) C-GIIF TDLITC
JFCOM JSIC
Wallops Island (2) SCSC SSDS
DISA Sky 7
Pentagon WARCAP
Dahlgren (2) CEDLIWSL
Dam Neck CDSA
Newport News NGC VASCIC
Sites in MD DC VA
Norfolk COMOPTEVFOR
McLean MITRE NCEL
Ft Huachuca JTRS
Bethpage NGC BCN
How an Air Force Customer Sees
Distributed Connectivity
Hardware-in-the-Loop (HWIL) Availability
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca JITC
Redstone (3) DTCC GMAN SED
Charleston (2) IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB
SIMAF
Bethpage NG BAMS
Whiteman B-2
MHPCC
PMRF Bldg 105
Sites in Hawaii
All linked by JMETC
29
Challenges Explained in Detail
30
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
Some TRMC Investments for
Achieving Common Vision
bull Central TampE Investment Program (CTEIP)
bull Test and Training Enabling Architecture (TENA)
bull Multi-Level Security-Joint Coalition Network Environment (MLS-
JCNE)
bull InterTEC
bull Result JITC-certified instrumentation usable in all aspects of TampE
bull Common Range Integrated Instrumentation System (CRIIS)
bull TampE SampT Program
bull TENA in a Resource Constrained Environment (TRCE)
bull NST Evaluation Capability Model (NECM)
bull JMETC Program
bull DoD corporate approach for distributed testing 19
What Can You Do to Be Ready for
2030
bull Maintain Information Assurance agreements for all facility
systems
bull Streamline data collection amp analysis at your facility
bull Get comfortable with DoDAF views
bull Look across the RDTampE the community to find common
problems that can be addressed with similar solutions
bull TENA-enable your instrumentation systems and test
command amp control
bull Get involved share your requirements with your Service
representatives and in TRMC users meetings
20
JMETC Users Group
bull Purpose is to focus on technical requirements and solutions relevant to current and future Distributed Testing needs
bull Technical and Management level representatives identify core infrastructure requirements and most importantly resolve issues
bull Quarterly meetings of 250-300 JMETC customers acquisition programs test events ranges LVC sites tools and network providers
21
Last JMETC Users Group Meeting bull June 2011 bull Location Mid-East Coast bull Tracks
bull User Requirements bull Information Assurance
Security bull Data Management bull InterTEC (Current amp Planned) bull Networking
bull An established forum for the Distributed Test Community to
o Identify core infrastructure requirements and use cases
o Identify investigate amp resolve issues
o Identify opportunities to collaborate
o Discuss available solutions tools and techniques
o Share lessons learned
Summary
bull We are becoming more efficient in how we execute TampE
bull We must get the biggest and best ldquobang for the buckrdquo that we can
bull TRMC and Services continue TampE investments to enable realization of the 2030 Vision
bull Align TampE (OSD and Services) investments to mitigate test risk areas
bull The Future of TampE will include Distributed Integrated Testing and Training and will enable near real time Test-Fix-Test
22
JMETC Program Points of Contact
23
JMETC Program Manager Chip Ferguson chipfergusonosdmil 703-601-5274
JMETC Principal Deputy PM Bruce Bailey brucebaileyosdmil 703-601-5208
JMETC Lead Operations Planning Marty Arnwine martemasarnwineosdmil 703-601-5215
JMETC Senior Technical Advisor George Rumford georgerumfordosdmil
703-601-5233
JMETC Lead Systems Engineer Ryan Norman ryannormanosdmil
703-601-5277
JMETC Lead Network Engineer Arjuna ldquoAJrdquo Pathmanathan ArjunaPathmanathanosdmil
703-601-5214
JMETC Website wwwjmetcorg
Backup Slides
25
FY10 JMETC Events
Customer Event Record Event Dates
Air Force JEFX 10-1Spirit Ice (B2 Data Link Test) October ndash November 2009
Navy BAMS LVC DE October ndash December 2009
Air Force Battlefield Airborne Comm Node (BACN) JUON
(DTOT)
November 2009 to September 2010
Air Force Agile Fire 10-2 January 2010
Air Force JEFX 10-23 FebruaryApril 2010
Joint Joint Surface Warfare JCTD February to September 2010
JS J8JIAMDO Joint Sensor Integration April to September 2010
Air Force B1-B Fully Integrated Data Link Testing April 2010
JFCOM J8489 (TEST WEEK) JCAS Distributed Test June 2010
JIAMDO (Navy Lead) CorrelationDecorrelation Interoperability Test
(CDIT) Integration Events (Continuous)
July to September 2010
Army (Lead) UAS in the National Airspace July to September 2010
Air Force Agile Fire 10-3 August 2010
Joint JITC Joint Interoperability Tests JIT 10-3 amp 11-1
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Joint Strike Fighter (JSF) JIAMDOJoint Track Manager
Brigade Combat Team (BCT)
Modernization
Multi-Function Adv Data Link (MADL) Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run 25
26
JMETC FY10 Accomplishments
JMETC Accomplishments
Supported 88 distinct customer test activities
Expanded network from 38 to 57 sites
ATIN and JTDL Networks transitioned to JMETC
Upgraded JMETC support applications and utilities to TENA R6
DIACAP Tiger Team report completed and recommendations being executed
Enhanced JMETC services and capabilities provided by leveraging InterTEC Services and Industry
Reuse Repository usability improvements
FY10 Example JMETC Customers
Joint Integrated Air amp Missile Defense Organization (JIAMDO)
Broad Area Communications Node (BACN) JUON
B1-B
Broad Area Maritime Surveillance System (BAMS)
Air-Ground Integrated Layer Exploration (AGILE)
Joint Interoperability Test Command (JITC)
Selected Benefits to the DoD
Integrated DT amp OT on a Joint Urgent Operational Need for the warfighter
Maximized usage of theater assets during limited maintenance windows
Improved Joint track information sharing to ensure interoperability of systems in theater operations
Coalition exchange and examination of real-time air picture data
Identification of Air Force Initiatives ready for warfighter transition
Investigated tactical UAS deployment in the National Airspace
Employment of Net-Enabled Weapons
JCAS immediate request amp end-to-end processes ldquoas-isrdquo characterization
Determined distributed system components were not ready for full live integration testing
Executed testing to support system-of-system interoperability certification
27
Customer Event Record Event Dates
Joint JIAMDO CorrelationDecorrelation Interoperability Test (CDIT)
Integration Events (Continuous) OCONUS October 2010
Navy UAS in NAS Runs for Record October 2010
Internal InterTEC Spiral 3 Systems Acceptance Test October - November 2010
Joint JITC Joint Interoperability Tests JIT 11-12345 (Continuous) October 2010 ndash September 2011
Air Force B1-B Fully Integrated Data Link Testing November 2010
Air Force AGILE Fire Phase III JEFX 2011 December 2010 - February 2011
Joint JTRS JPO -- JTRS Ground Mobile Radio January 2011
Navy Broad Area Maritime Surveillance (BAMS) (Continuous) January ndash September 2011
Joint Joint Track Manager Capability Demonstration (Continuous) January ndash September 2011
Joint JSF Initial MampS Interoperability (Continuous) February ndash March 2011
Air Force JSTARS Interoperability Test May 2011
Joint JS J8JIAMDO Joint Sensor Integration June ndash August 2011
FY11 JMETC Events
(More to Come)
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Global Hawk GATOR
Brigade Combat Team (BCT)
Modernization F-22 FY12 Testing Planned Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run
28
JMETC Connectivity
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca (2) JITC JTDL
Redstone (7) SMDC
RTC DTCC GMAN
SED Patriot THAAD FAAD SIV
Charleston (2)IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB SIMAF
As of 16 Mar 2011
0
10
20
30
40
50
60
70
FY06 FY07 FY08 FY09 FY10 FY11
JMETC Site Expansion
Sites
Dedicated trusted connectivity on SDREN (part of the GIG)
Encrypted for Secret ndash System High
DISA-registered IP address space
Active monitoring of network performance
Capable of supporting multiple simultaneous test events Bethpage NGC BAMS
Whiteman B-2
Eglin AFB (4)AOC DTF GWEF KHILS
Hurlburt Field C2DAC
Sites in Gulf Range
Functional Sites 61
New Sites Planned 8
Connection Points to Other Networks 5
Ft Greely CRTC
Site in Alaska
MHPCC
PMRF Bldg 105
Sites in Hawaii
Camp PendletonMCTSSA
China Lake (3)AV-8B FA-18IBAR
Edwards Ridley
Corona NSWC
El Segundo NGC B-2
Sites in SoCal
Point Loma (2)RLBTSSSC-PAC 59140 SSC-PAC HAIPE
Point Mugu (2)ITEC EA-6B
West Agg Rtr
Rancho Bernardo NGC BAMS
Pax River (5)ESTEL E2CD MCLACETEF SAIL ATR JMETC SYSCONEast Agg Rtr
Aberdeen ACCN
Langley (2) C-GIIF TDLITC
JFCOM JSIC
Wallops Island (2) SCSC SSDS
DISA Sky 7
Pentagon WARCAP
Dahlgren (2) CEDLIWSL
Dam Neck CDSA
Newport News NGC VASCIC
Sites in MD DC VA
Norfolk COMOPTEVFOR
McLean MITRE NCEL
Ft Huachuca JTRS
Bethpage NGC BCN
How an Air Force Customer Sees
Distributed Connectivity
Hardware-in-the-Loop (HWIL) Availability
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca JITC
Redstone (3) DTCC GMAN SED
Charleston (2) IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB
SIMAF
Bethpage NG BAMS
Whiteman B-2
MHPCC
PMRF Bldg 105
Sites in Hawaii
All linked by JMETC
29
Challenges Explained in Detail
30
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
What Can You Do to Be Ready for
2030
bull Maintain Information Assurance agreements for all facility
systems
bull Streamline data collection amp analysis at your facility
bull Get comfortable with DoDAF views
bull Look across the RDTampE the community to find common
problems that can be addressed with similar solutions
bull TENA-enable your instrumentation systems and test
command amp control
bull Get involved share your requirements with your Service
representatives and in TRMC users meetings
20
JMETC Users Group
bull Purpose is to focus on technical requirements and solutions relevant to current and future Distributed Testing needs
bull Technical and Management level representatives identify core infrastructure requirements and most importantly resolve issues
bull Quarterly meetings of 250-300 JMETC customers acquisition programs test events ranges LVC sites tools and network providers
21
Last JMETC Users Group Meeting bull June 2011 bull Location Mid-East Coast bull Tracks
bull User Requirements bull Information Assurance
Security bull Data Management bull InterTEC (Current amp Planned) bull Networking
bull An established forum for the Distributed Test Community to
o Identify core infrastructure requirements and use cases
o Identify investigate amp resolve issues
o Identify opportunities to collaborate
o Discuss available solutions tools and techniques
o Share lessons learned
Summary
bull We are becoming more efficient in how we execute TampE
bull We must get the biggest and best ldquobang for the buckrdquo that we can
bull TRMC and Services continue TampE investments to enable realization of the 2030 Vision
bull Align TampE (OSD and Services) investments to mitigate test risk areas
bull The Future of TampE will include Distributed Integrated Testing and Training and will enable near real time Test-Fix-Test
22
JMETC Program Points of Contact
23
JMETC Program Manager Chip Ferguson chipfergusonosdmil 703-601-5274
JMETC Principal Deputy PM Bruce Bailey brucebaileyosdmil 703-601-5208
JMETC Lead Operations Planning Marty Arnwine martemasarnwineosdmil 703-601-5215
JMETC Senior Technical Advisor George Rumford georgerumfordosdmil
703-601-5233
JMETC Lead Systems Engineer Ryan Norman ryannormanosdmil
703-601-5277
JMETC Lead Network Engineer Arjuna ldquoAJrdquo Pathmanathan ArjunaPathmanathanosdmil
703-601-5214
JMETC Website wwwjmetcorg
Backup Slides
25
FY10 JMETC Events
Customer Event Record Event Dates
Air Force JEFX 10-1Spirit Ice (B2 Data Link Test) October ndash November 2009
Navy BAMS LVC DE October ndash December 2009
Air Force Battlefield Airborne Comm Node (BACN) JUON
(DTOT)
November 2009 to September 2010
Air Force Agile Fire 10-2 January 2010
Air Force JEFX 10-23 FebruaryApril 2010
Joint Joint Surface Warfare JCTD February to September 2010
JS J8JIAMDO Joint Sensor Integration April to September 2010
Air Force B1-B Fully Integrated Data Link Testing April 2010
JFCOM J8489 (TEST WEEK) JCAS Distributed Test June 2010
JIAMDO (Navy Lead) CorrelationDecorrelation Interoperability Test
(CDIT) Integration Events (Continuous)
July to September 2010
Army (Lead) UAS in the National Airspace July to September 2010
Air Force Agile Fire 10-3 August 2010
Joint JITC Joint Interoperability Tests JIT 10-3 amp 11-1
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Joint Strike Fighter (JSF) JIAMDOJoint Track Manager
Brigade Combat Team (BCT)
Modernization
Multi-Function Adv Data Link (MADL) Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run 25
26
JMETC FY10 Accomplishments
JMETC Accomplishments
Supported 88 distinct customer test activities
Expanded network from 38 to 57 sites
ATIN and JTDL Networks transitioned to JMETC
Upgraded JMETC support applications and utilities to TENA R6
DIACAP Tiger Team report completed and recommendations being executed
Enhanced JMETC services and capabilities provided by leveraging InterTEC Services and Industry
Reuse Repository usability improvements
FY10 Example JMETC Customers
Joint Integrated Air amp Missile Defense Organization (JIAMDO)
Broad Area Communications Node (BACN) JUON
B1-B
Broad Area Maritime Surveillance System (BAMS)
Air-Ground Integrated Layer Exploration (AGILE)
Joint Interoperability Test Command (JITC)
Selected Benefits to the DoD
Integrated DT amp OT on a Joint Urgent Operational Need for the warfighter
Maximized usage of theater assets during limited maintenance windows
Improved Joint track information sharing to ensure interoperability of systems in theater operations
Coalition exchange and examination of real-time air picture data
Identification of Air Force Initiatives ready for warfighter transition
Investigated tactical UAS deployment in the National Airspace
Employment of Net-Enabled Weapons
JCAS immediate request amp end-to-end processes ldquoas-isrdquo characterization
Determined distributed system components were not ready for full live integration testing
Executed testing to support system-of-system interoperability certification
27
Customer Event Record Event Dates
Joint JIAMDO CorrelationDecorrelation Interoperability Test (CDIT)
Integration Events (Continuous) OCONUS October 2010
Navy UAS in NAS Runs for Record October 2010
Internal InterTEC Spiral 3 Systems Acceptance Test October - November 2010
Joint JITC Joint Interoperability Tests JIT 11-12345 (Continuous) October 2010 ndash September 2011
Air Force B1-B Fully Integrated Data Link Testing November 2010
Air Force AGILE Fire Phase III JEFX 2011 December 2010 - February 2011
Joint JTRS JPO -- JTRS Ground Mobile Radio January 2011
Navy Broad Area Maritime Surveillance (BAMS) (Continuous) January ndash September 2011
Joint Joint Track Manager Capability Demonstration (Continuous) January ndash September 2011
Joint JSF Initial MampS Interoperability (Continuous) February ndash March 2011
Air Force JSTARS Interoperability Test May 2011
Joint JS J8JIAMDO Joint Sensor Integration June ndash August 2011
FY11 JMETC Events
(More to Come)
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Global Hawk GATOR
Brigade Combat Team (BCT)
Modernization F-22 FY12 Testing Planned Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run
28
JMETC Connectivity
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca (2) JITC JTDL
Redstone (7) SMDC
RTC DTCC GMAN
SED Patriot THAAD FAAD SIV
Charleston (2)IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB SIMAF
As of 16 Mar 2011
0
10
20
30
40
50
60
70
FY06 FY07 FY08 FY09 FY10 FY11
JMETC Site Expansion
Sites
Dedicated trusted connectivity on SDREN (part of the GIG)
Encrypted for Secret ndash System High
DISA-registered IP address space
Active monitoring of network performance
Capable of supporting multiple simultaneous test events Bethpage NGC BAMS
Whiteman B-2
Eglin AFB (4)AOC DTF GWEF KHILS
Hurlburt Field C2DAC
Sites in Gulf Range
Functional Sites 61
New Sites Planned 8
Connection Points to Other Networks 5
Ft Greely CRTC
Site in Alaska
MHPCC
PMRF Bldg 105
Sites in Hawaii
Camp PendletonMCTSSA
China Lake (3)AV-8B FA-18IBAR
Edwards Ridley
Corona NSWC
El Segundo NGC B-2
Sites in SoCal
Point Loma (2)RLBTSSSC-PAC 59140 SSC-PAC HAIPE
Point Mugu (2)ITEC EA-6B
West Agg Rtr
Rancho Bernardo NGC BAMS
Pax River (5)ESTEL E2CD MCLACETEF SAIL ATR JMETC SYSCONEast Agg Rtr
Aberdeen ACCN
Langley (2) C-GIIF TDLITC
JFCOM JSIC
Wallops Island (2) SCSC SSDS
DISA Sky 7
Pentagon WARCAP
Dahlgren (2) CEDLIWSL
Dam Neck CDSA
Newport News NGC VASCIC
Sites in MD DC VA
Norfolk COMOPTEVFOR
McLean MITRE NCEL
Ft Huachuca JTRS
Bethpage NGC BCN
How an Air Force Customer Sees
Distributed Connectivity
Hardware-in-the-Loop (HWIL) Availability
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca JITC
Redstone (3) DTCC GMAN SED
Charleston (2) IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB
SIMAF
Bethpage NG BAMS
Whiteman B-2
MHPCC
PMRF Bldg 105
Sites in Hawaii
All linked by JMETC
29
Challenges Explained in Detail
30
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
JMETC Users Group
bull Purpose is to focus on technical requirements and solutions relevant to current and future Distributed Testing needs
bull Technical and Management level representatives identify core infrastructure requirements and most importantly resolve issues
bull Quarterly meetings of 250-300 JMETC customers acquisition programs test events ranges LVC sites tools and network providers
21
Last JMETC Users Group Meeting bull June 2011 bull Location Mid-East Coast bull Tracks
bull User Requirements bull Information Assurance
Security bull Data Management bull InterTEC (Current amp Planned) bull Networking
bull An established forum for the Distributed Test Community to
o Identify core infrastructure requirements and use cases
o Identify investigate amp resolve issues
o Identify opportunities to collaborate
o Discuss available solutions tools and techniques
o Share lessons learned
Summary
bull We are becoming more efficient in how we execute TampE
bull We must get the biggest and best ldquobang for the buckrdquo that we can
bull TRMC and Services continue TampE investments to enable realization of the 2030 Vision
bull Align TampE (OSD and Services) investments to mitigate test risk areas
bull The Future of TampE will include Distributed Integrated Testing and Training and will enable near real time Test-Fix-Test
22
JMETC Program Points of Contact
23
JMETC Program Manager Chip Ferguson chipfergusonosdmil 703-601-5274
JMETC Principal Deputy PM Bruce Bailey brucebaileyosdmil 703-601-5208
JMETC Lead Operations Planning Marty Arnwine martemasarnwineosdmil 703-601-5215
JMETC Senior Technical Advisor George Rumford georgerumfordosdmil
703-601-5233
JMETC Lead Systems Engineer Ryan Norman ryannormanosdmil
703-601-5277
JMETC Lead Network Engineer Arjuna ldquoAJrdquo Pathmanathan ArjunaPathmanathanosdmil
703-601-5214
JMETC Website wwwjmetcorg
Backup Slides
25
FY10 JMETC Events
Customer Event Record Event Dates
Air Force JEFX 10-1Spirit Ice (B2 Data Link Test) October ndash November 2009
Navy BAMS LVC DE October ndash December 2009
Air Force Battlefield Airborne Comm Node (BACN) JUON
(DTOT)
November 2009 to September 2010
Air Force Agile Fire 10-2 January 2010
Air Force JEFX 10-23 FebruaryApril 2010
Joint Joint Surface Warfare JCTD February to September 2010
JS J8JIAMDO Joint Sensor Integration April to September 2010
Air Force B1-B Fully Integrated Data Link Testing April 2010
JFCOM J8489 (TEST WEEK) JCAS Distributed Test June 2010
JIAMDO (Navy Lead) CorrelationDecorrelation Interoperability Test
(CDIT) Integration Events (Continuous)
July to September 2010
Army (Lead) UAS in the National Airspace July to September 2010
Air Force Agile Fire 10-3 August 2010
Joint JITC Joint Interoperability Tests JIT 10-3 amp 11-1
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Joint Strike Fighter (JSF) JIAMDOJoint Track Manager
Brigade Combat Team (BCT)
Modernization
Multi-Function Adv Data Link (MADL) Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run 25
26
JMETC FY10 Accomplishments
JMETC Accomplishments
Supported 88 distinct customer test activities
Expanded network from 38 to 57 sites
ATIN and JTDL Networks transitioned to JMETC
Upgraded JMETC support applications and utilities to TENA R6
DIACAP Tiger Team report completed and recommendations being executed
Enhanced JMETC services and capabilities provided by leveraging InterTEC Services and Industry
Reuse Repository usability improvements
FY10 Example JMETC Customers
Joint Integrated Air amp Missile Defense Organization (JIAMDO)
Broad Area Communications Node (BACN) JUON
B1-B
Broad Area Maritime Surveillance System (BAMS)
Air-Ground Integrated Layer Exploration (AGILE)
Joint Interoperability Test Command (JITC)
Selected Benefits to the DoD
Integrated DT amp OT on a Joint Urgent Operational Need for the warfighter
Maximized usage of theater assets during limited maintenance windows
Improved Joint track information sharing to ensure interoperability of systems in theater operations
Coalition exchange and examination of real-time air picture data
Identification of Air Force Initiatives ready for warfighter transition
Investigated tactical UAS deployment in the National Airspace
Employment of Net-Enabled Weapons
JCAS immediate request amp end-to-end processes ldquoas-isrdquo characterization
Determined distributed system components were not ready for full live integration testing
Executed testing to support system-of-system interoperability certification
27
Customer Event Record Event Dates
Joint JIAMDO CorrelationDecorrelation Interoperability Test (CDIT)
Integration Events (Continuous) OCONUS October 2010
Navy UAS in NAS Runs for Record October 2010
Internal InterTEC Spiral 3 Systems Acceptance Test October - November 2010
Joint JITC Joint Interoperability Tests JIT 11-12345 (Continuous) October 2010 ndash September 2011
Air Force B1-B Fully Integrated Data Link Testing November 2010
Air Force AGILE Fire Phase III JEFX 2011 December 2010 - February 2011
Joint JTRS JPO -- JTRS Ground Mobile Radio January 2011
Navy Broad Area Maritime Surveillance (BAMS) (Continuous) January ndash September 2011
Joint Joint Track Manager Capability Demonstration (Continuous) January ndash September 2011
Joint JSF Initial MampS Interoperability (Continuous) February ndash March 2011
Air Force JSTARS Interoperability Test May 2011
Joint JS J8JIAMDO Joint Sensor Integration June ndash August 2011
FY11 JMETC Events
(More to Come)
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Global Hawk GATOR
Brigade Combat Team (BCT)
Modernization F-22 FY12 Testing Planned Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run
28
JMETC Connectivity
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca (2) JITC JTDL
Redstone (7) SMDC
RTC DTCC GMAN
SED Patriot THAAD FAAD SIV
Charleston (2)IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB SIMAF
As of 16 Mar 2011
0
10
20
30
40
50
60
70
FY06 FY07 FY08 FY09 FY10 FY11
JMETC Site Expansion
Sites
Dedicated trusted connectivity on SDREN (part of the GIG)
Encrypted for Secret ndash System High
DISA-registered IP address space
Active monitoring of network performance
Capable of supporting multiple simultaneous test events Bethpage NGC BAMS
Whiteman B-2
Eglin AFB (4)AOC DTF GWEF KHILS
Hurlburt Field C2DAC
Sites in Gulf Range
Functional Sites 61
New Sites Planned 8
Connection Points to Other Networks 5
Ft Greely CRTC
Site in Alaska
MHPCC
PMRF Bldg 105
Sites in Hawaii
Camp PendletonMCTSSA
China Lake (3)AV-8B FA-18IBAR
Edwards Ridley
Corona NSWC
El Segundo NGC B-2
Sites in SoCal
Point Loma (2)RLBTSSSC-PAC 59140 SSC-PAC HAIPE
Point Mugu (2)ITEC EA-6B
West Agg Rtr
Rancho Bernardo NGC BAMS
Pax River (5)ESTEL E2CD MCLACETEF SAIL ATR JMETC SYSCONEast Agg Rtr
Aberdeen ACCN
Langley (2) C-GIIF TDLITC
JFCOM JSIC
Wallops Island (2) SCSC SSDS
DISA Sky 7
Pentagon WARCAP
Dahlgren (2) CEDLIWSL
Dam Neck CDSA
Newport News NGC VASCIC
Sites in MD DC VA
Norfolk COMOPTEVFOR
McLean MITRE NCEL
Ft Huachuca JTRS
Bethpage NGC BCN
How an Air Force Customer Sees
Distributed Connectivity
Hardware-in-the-Loop (HWIL) Availability
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca JITC
Redstone (3) DTCC GMAN SED
Charleston (2) IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB
SIMAF
Bethpage NG BAMS
Whiteman B-2
MHPCC
PMRF Bldg 105
Sites in Hawaii
All linked by JMETC
29
Challenges Explained in Detail
30
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
Summary
bull We are becoming more efficient in how we execute TampE
bull We must get the biggest and best ldquobang for the buckrdquo that we can
bull TRMC and Services continue TampE investments to enable realization of the 2030 Vision
bull Align TampE (OSD and Services) investments to mitigate test risk areas
bull The Future of TampE will include Distributed Integrated Testing and Training and will enable near real time Test-Fix-Test
22
JMETC Program Points of Contact
23
JMETC Program Manager Chip Ferguson chipfergusonosdmil 703-601-5274
JMETC Principal Deputy PM Bruce Bailey brucebaileyosdmil 703-601-5208
JMETC Lead Operations Planning Marty Arnwine martemasarnwineosdmil 703-601-5215
JMETC Senior Technical Advisor George Rumford georgerumfordosdmil
703-601-5233
JMETC Lead Systems Engineer Ryan Norman ryannormanosdmil
703-601-5277
JMETC Lead Network Engineer Arjuna ldquoAJrdquo Pathmanathan ArjunaPathmanathanosdmil
703-601-5214
JMETC Website wwwjmetcorg
Backup Slides
25
FY10 JMETC Events
Customer Event Record Event Dates
Air Force JEFX 10-1Spirit Ice (B2 Data Link Test) October ndash November 2009
Navy BAMS LVC DE October ndash December 2009
Air Force Battlefield Airborne Comm Node (BACN) JUON
(DTOT)
November 2009 to September 2010
Air Force Agile Fire 10-2 January 2010
Air Force JEFX 10-23 FebruaryApril 2010
Joint Joint Surface Warfare JCTD February to September 2010
JS J8JIAMDO Joint Sensor Integration April to September 2010
Air Force B1-B Fully Integrated Data Link Testing April 2010
JFCOM J8489 (TEST WEEK) JCAS Distributed Test June 2010
JIAMDO (Navy Lead) CorrelationDecorrelation Interoperability Test
(CDIT) Integration Events (Continuous)
July to September 2010
Army (Lead) UAS in the National Airspace July to September 2010
Air Force Agile Fire 10-3 August 2010
Joint JITC Joint Interoperability Tests JIT 10-3 amp 11-1
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Joint Strike Fighter (JSF) JIAMDOJoint Track Manager
Brigade Combat Team (BCT)
Modernization
Multi-Function Adv Data Link (MADL) Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run 25
26
JMETC FY10 Accomplishments
JMETC Accomplishments
Supported 88 distinct customer test activities
Expanded network from 38 to 57 sites
ATIN and JTDL Networks transitioned to JMETC
Upgraded JMETC support applications and utilities to TENA R6
DIACAP Tiger Team report completed and recommendations being executed
Enhanced JMETC services and capabilities provided by leveraging InterTEC Services and Industry
Reuse Repository usability improvements
FY10 Example JMETC Customers
Joint Integrated Air amp Missile Defense Organization (JIAMDO)
Broad Area Communications Node (BACN) JUON
B1-B
Broad Area Maritime Surveillance System (BAMS)
Air-Ground Integrated Layer Exploration (AGILE)
Joint Interoperability Test Command (JITC)
Selected Benefits to the DoD
Integrated DT amp OT on a Joint Urgent Operational Need for the warfighter
Maximized usage of theater assets during limited maintenance windows
Improved Joint track information sharing to ensure interoperability of systems in theater operations
Coalition exchange and examination of real-time air picture data
Identification of Air Force Initiatives ready for warfighter transition
Investigated tactical UAS deployment in the National Airspace
Employment of Net-Enabled Weapons
JCAS immediate request amp end-to-end processes ldquoas-isrdquo characterization
Determined distributed system components were not ready for full live integration testing
Executed testing to support system-of-system interoperability certification
27
Customer Event Record Event Dates
Joint JIAMDO CorrelationDecorrelation Interoperability Test (CDIT)
Integration Events (Continuous) OCONUS October 2010
Navy UAS in NAS Runs for Record October 2010
Internal InterTEC Spiral 3 Systems Acceptance Test October - November 2010
Joint JITC Joint Interoperability Tests JIT 11-12345 (Continuous) October 2010 ndash September 2011
Air Force B1-B Fully Integrated Data Link Testing November 2010
Air Force AGILE Fire Phase III JEFX 2011 December 2010 - February 2011
Joint JTRS JPO -- JTRS Ground Mobile Radio January 2011
Navy Broad Area Maritime Surveillance (BAMS) (Continuous) January ndash September 2011
Joint Joint Track Manager Capability Demonstration (Continuous) January ndash September 2011
Joint JSF Initial MampS Interoperability (Continuous) February ndash March 2011
Air Force JSTARS Interoperability Test May 2011
Joint JS J8JIAMDO Joint Sensor Integration June ndash August 2011
FY11 JMETC Events
(More to Come)
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Global Hawk GATOR
Brigade Combat Team (BCT)
Modernization F-22 FY12 Testing Planned Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run
28
JMETC Connectivity
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca (2) JITC JTDL
Redstone (7) SMDC
RTC DTCC GMAN
SED Patriot THAAD FAAD SIV
Charleston (2)IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB SIMAF
As of 16 Mar 2011
0
10
20
30
40
50
60
70
FY06 FY07 FY08 FY09 FY10 FY11
JMETC Site Expansion
Sites
Dedicated trusted connectivity on SDREN (part of the GIG)
Encrypted for Secret ndash System High
DISA-registered IP address space
Active monitoring of network performance
Capable of supporting multiple simultaneous test events Bethpage NGC BAMS
Whiteman B-2
Eglin AFB (4)AOC DTF GWEF KHILS
Hurlburt Field C2DAC
Sites in Gulf Range
Functional Sites 61
New Sites Planned 8
Connection Points to Other Networks 5
Ft Greely CRTC
Site in Alaska
MHPCC
PMRF Bldg 105
Sites in Hawaii
Camp PendletonMCTSSA
China Lake (3)AV-8B FA-18IBAR
Edwards Ridley
Corona NSWC
El Segundo NGC B-2
Sites in SoCal
Point Loma (2)RLBTSSSC-PAC 59140 SSC-PAC HAIPE
Point Mugu (2)ITEC EA-6B
West Agg Rtr
Rancho Bernardo NGC BAMS
Pax River (5)ESTEL E2CD MCLACETEF SAIL ATR JMETC SYSCONEast Agg Rtr
Aberdeen ACCN
Langley (2) C-GIIF TDLITC
JFCOM JSIC
Wallops Island (2) SCSC SSDS
DISA Sky 7
Pentagon WARCAP
Dahlgren (2) CEDLIWSL
Dam Neck CDSA
Newport News NGC VASCIC
Sites in MD DC VA
Norfolk COMOPTEVFOR
McLean MITRE NCEL
Ft Huachuca JTRS
Bethpage NGC BCN
How an Air Force Customer Sees
Distributed Connectivity
Hardware-in-the-Loop (HWIL) Availability
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca JITC
Redstone (3) DTCC GMAN SED
Charleston (2) IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB
SIMAF
Bethpage NG BAMS
Whiteman B-2
MHPCC
PMRF Bldg 105
Sites in Hawaii
All linked by JMETC
29
Challenges Explained in Detail
30
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
JMETC Program Points of Contact
23
JMETC Program Manager Chip Ferguson chipfergusonosdmil 703-601-5274
JMETC Principal Deputy PM Bruce Bailey brucebaileyosdmil 703-601-5208
JMETC Lead Operations Planning Marty Arnwine martemasarnwineosdmil 703-601-5215
JMETC Senior Technical Advisor George Rumford georgerumfordosdmil
703-601-5233
JMETC Lead Systems Engineer Ryan Norman ryannormanosdmil
703-601-5277
JMETC Lead Network Engineer Arjuna ldquoAJrdquo Pathmanathan ArjunaPathmanathanosdmil
703-601-5214
JMETC Website wwwjmetcorg
Backup Slides
25
FY10 JMETC Events
Customer Event Record Event Dates
Air Force JEFX 10-1Spirit Ice (B2 Data Link Test) October ndash November 2009
Navy BAMS LVC DE October ndash December 2009
Air Force Battlefield Airborne Comm Node (BACN) JUON
(DTOT)
November 2009 to September 2010
Air Force Agile Fire 10-2 January 2010
Air Force JEFX 10-23 FebruaryApril 2010
Joint Joint Surface Warfare JCTD February to September 2010
JS J8JIAMDO Joint Sensor Integration April to September 2010
Air Force B1-B Fully Integrated Data Link Testing April 2010
JFCOM J8489 (TEST WEEK) JCAS Distributed Test June 2010
JIAMDO (Navy Lead) CorrelationDecorrelation Interoperability Test
(CDIT) Integration Events (Continuous)
July to September 2010
Army (Lead) UAS in the National Airspace July to September 2010
Air Force Agile Fire 10-3 August 2010
Joint JITC Joint Interoperability Tests JIT 10-3 amp 11-1
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Joint Strike Fighter (JSF) JIAMDOJoint Track Manager
Brigade Combat Team (BCT)
Modernization
Multi-Function Adv Data Link (MADL) Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run 25
26
JMETC FY10 Accomplishments
JMETC Accomplishments
Supported 88 distinct customer test activities
Expanded network from 38 to 57 sites
ATIN and JTDL Networks transitioned to JMETC
Upgraded JMETC support applications and utilities to TENA R6
DIACAP Tiger Team report completed and recommendations being executed
Enhanced JMETC services and capabilities provided by leveraging InterTEC Services and Industry
Reuse Repository usability improvements
FY10 Example JMETC Customers
Joint Integrated Air amp Missile Defense Organization (JIAMDO)
Broad Area Communications Node (BACN) JUON
B1-B
Broad Area Maritime Surveillance System (BAMS)
Air-Ground Integrated Layer Exploration (AGILE)
Joint Interoperability Test Command (JITC)
Selected Benefits to the DoD
Integrated DT amp OT on a Joint Urgent Operational Need for the warfighter
Maximized usage of theater assets during limited maintenance windows
Improved Joint track information sharing to ensure interoperability of systems in theater operations
Coalition exchange and examination of real-time air picture data
Identification of Air Force Initiatives ready for warfighter transition
Investigated tactical UAS deployment in the National Airspace
Employment of Net-Enabled Weapons
JCAS immediate request amp end-to-end processes ldquoas-isrdquo characterization
Determined distributed system components were not ready for full live integration testing
Executed testing to support system-of-system interoperability certification
27
Customer Event Record Event Dates
Joint JIAMDO CorrelationDecorrelation Interoperability Test (CDIT)
Integration Events (Continuous) OCONUS October 2010
Navy UAS in NAS Runs for Record October 2010
Internal InterTEC Spiral 3 Systems Acceptance Test October - November 2010
Joint JITC Joint Interoperability Tests JIT 11-12345 (Continuous) October 2010 ndash September 2011
Air Force B1-B Fully Integrated Data Link Testing November 2010
Air Force AGILE Fire Phase III JEFX 2011 December 2010 - February 2011
Joint JTRS JPO -- JTRS Ground Mobile Radio January 2011
Navy Broad Area Maritime Surveillance (BAMS) (Continuous) January ndash September 2011
Joint Joint Track Manager Capability Demonstration (Continuous) January ndash September 2011
Joint JSF Initial MampS Interoperability (Continuous) February ndash March 2011
Air Force JSTARS Interoperability Test May 2011
Joint JS J8JIAMDO Joint Sensor Integration June ndash August 2011
FY11 JMETC Events
(More to Come)
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Global Hawk GATOR
Brigade Combat Team (BCT)
Modernization F-22 FY12 Testing Planned Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run
28
JMETC Connectivity
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca (2) JITC JTDL
Redstone (7) SMDC
RTC DTCC GMAN
SED Patriot THAAD FAAD SIV
Charleston (2)IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB SIMAF
As of 16 Mar 2011
0
10
20
30
40
50
60
70
FY06 FY07 FY08 FY09 FY10 FY11
JMETC Site Expansion
Sites
Dedicated trusted connectivity on SDREN (part of the GIG)
Encrypted for Secret ndash System High
DISA-registered IP address space
Active monitoring of network performance
Capable of supporting multiple simultaneous test events Bethpage NGC BAMS
Whiteman B-2
Eglin AFB (4)AOC DTF GWEF KHILS
Hurlburt Field C2DAC
Sites in Gulf Range
Functional Sites 61
New Sites Planned 8
Connection Points to Other Networks 5
Ft Greely CRTC
Site in Alaska
MHPCC
PMRF Bldg 105
Sites in Hawaii
Camp PendletonMCTSSA
China Lake (3)AV-8B FA-18IBAR
Edwards Ridley
Corona NSWC
El Segundo NGC B-2
Sites in SoCal
Point Loma (2)RLBTSSSC-PAC 59140 SSC-PAC HAIPE
Point Mugu (2)ITEC EA-6B
West Agg Rtr
Rancho Bernardo NGC BAMS
Pax River (5)ESTEL E2CD MCLACETEF SAIL ATR JMETC SYSCONEast Agg Rtr
Aberdeen ACCN
Langley (2) C-GIIF TDLITC
JFCOM JSIC
Wallops Island (2) SCSC SSDS
DISA Sky 7
Pentagon WARCAP
Dahlgren (2) CEDLIWSL
Dam Neck CDSA
Newport News NGC VASCIC
Sites in MD DC VA
Norfolk COMOPTEVFOR
McLean MITRE NCEL
Ft Huachuca JTRS
Bethpage NGC BCN
How an Air Force Customer Sees
Distributed Connectivity
Hardware-in-the-Loop (HWIL) Availability
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca JITC
Redstone (3) DTCC GMAN SED
Charleston (2) IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB
SIMAF
Bethpage NG BAMS
Whiteman B-2
MHPCC
PMRF Bldg 105
Sites in Hawaii
All linked by JMETC
29
Challenges Explained in Detail
30
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
Backup Slides
25
FY10 JMETC Events
Customer Event Record Event Dates
Air Force JEFX 10-1Spirit Ice (B2 Data Link Test) October ndash November 2009
Navy BAMS LVC DE October ndash December 2009
Air Force Battlefield Airborne Comm Node (BACN) JUON
(DTOT)
November 2009 to September 2010
Air Force Agile Fire 10-2 January 2010
Air Force JEFX 10-23 FebruaryApril 2010
Joint Joint Surface Warfare JCTD February to September 2010
JS J8JIAMDO Joint Sensor Integration April to September 2010
Air Force B1-B Fully Integrated Data Link Testing April 2010
JFCOM J8489 (TEST WEEK) JCAS Distributed Test June 2010
JIAMDO (Navy Lead) CorrelationDecorrelation Interoperability Test
(CDIT) Integration Events (Continuous)
July to September 2010
Army (Lead) UAS in the National Airspace July to September 2010
Air Force Agile Fire 10-3 August 2010
Joint JITC Joint Interoperability Tests JIT 10-3 amp 11-1
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Joint Strike Fighter (JSF) JIAMDOJoint Track Manager
Brigade Combat Team (BCT)
Modernization
Multi-Function Adv Data Link (MADL) Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run 25
26
JMETC FY10 Accomplishments
JMETC Accomplishments
Supported 88 distinct customer test activities
Expanded network from 38 to 57 sites
ATIN and JTDL Networks transitioned to JMETC
Upgraded JMETC support applications and utilities to TENA R6
DIACAP Tiger Team report completed and recommendations being executed
Enhanced JMETC services and capabilities provided by leveraging InterTEC Services and Industry
Reuse Repository usability improvements
FY10 Example JMETC Customers
Joint Integrated Air amp Missile Defense Organization (JIAMDO)
Broad Area Communications Node (BACN) JUON
B1-B
Broad Area Maritime Surveillance System (BAMS)
Air-Ground Integrated Layer Exploration (AGILE)
Joint Interoperability Test Command (JITC)
Selected Benefits to the DoD
Integrated DT amp OT on a Joint Urgent Operational Need for the warfighter
Maximized usage of theater assets during limited maintenance windows
Improved Joint track information sharing to ensure interoperability of systems in theater operations
Coalition exchange and examination of real-time air picture data
Identification of Air Force Initiatives ready for warfighter transition
Investigated tactical UAS deployment in the National Airspace
Employment of Net-Enabled Weapons
JCAS immediate request amp end-to-end processes ldquoas-isrdquo characterization
Determined distributed system components were not ready for full live integration testing
Executed testing to support system-of-system interoperability certification
27
Customer Event Record Event Dates
Joint JIAMDO CorrelationDecorrelation Interoperability Test (CDIT)
Integration Events (Continuous) OCONUS October 2010
Navy UAS in NAS Runs for Record October 2010
Internal InterTEC Spiral 3 Systems Acceptance Test October - November 2010
Joint JITC Joint Interoperability Tests JIT 11-12345 (Continuous) October 2010 ndash September 2011
Air Force B1-B Fully Integrated Data Link Testing November 2010
Air Force AGILE Fire Phase III JEFX 2011 December 2010 - February 2011
Joint JTRS JPO -- JTRS Ground Mobile Radio January 2011
Navy Broad Area Maritime Surveillance (BAMS) (Continuous) January ndash September 2011
Joint Joint Track Manager Capability Demonstration (Continuous) January ndash September 2011
Joint JSF Initial MampS Interoperability (Continuous) February ndash March 2011
Air Force JSTARS Interoperability Test May 2011
Joint JS J8JIAMDO Joint Sensor Integration June ndash August 2011
FY11 JMETC Events
(More to Come)
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Global Hawk GATOR
Brigade Combat Team (BCT)
Modernization F-22 FY12 Testing Planned Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run
28
JMETC Connectivity
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca (2) JITC JTDL
Redstone (7) SMDC
RTC DTCC GMAN
SED Patriot THAAD FAAD SIV
Charleston (2)IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB SIMAF
As of 16 Mar 2011
0
10
20
30
40
50
60
70
FY06 FY07 FY08 FY09 FY10 FY11
JMETC Site Expansion
Sites
Dedicated trusted connectivity on SDREN (part of the GIG)
Encrypted for Secret ndash System High
DISA-registered IP address space
Active monitoring of network performance
Capable of supporting multiple simultaneous test events Bethpage NGC BAMS
Whiteman B-2
Eglin AFB (4)AOC DTF GWEF KHILS
Hurlburt Field C2DAC
Sites in Gulf Range
Functional Sites 61
New Sites Planned 8
Connection Points to Other Networks 5
Ft Greely CRTC
Site in Alaska
MHPCC
PMRF Bldg 105
Sites in Hawaii
Camp PendletonMCTSSA
China Lake (3)AV-8B FA-18IBAR
Edwards Ridley
Corona NSWC
El Segundo NGC B-2
Sites in SoCal
Point Loma (2)RLBTSSSC-PAC 59140 SSC-PAC HAIPE
Point Mugu (2)ITEC EA-6B
West Agg Rtr
Rancho Bernardo NGC BAMS
Pax River (5)ESTEL E2CD MCLACETEF SAIL ATR JMETC SYSCONEast Agg Rtr
Aberdeen ACCN
Langley (2) C-GIIF TDLITC
JFCOM JSIC
Wallops Island (2) SCSC SSDS
DISA Sky 7
Pentagon WARCAP
Dahlgren (2) CEDLIWSL
Dam Neck CDSA
Newport News NGC VASCIC
Sites in MD DC VA
Norfolk COMOPTEVFOR
McLean MITRE NCEL
Ft Huachuca JTRS
Bethpage NGC BCN
How an Air Force Customer Sees
Distributed Connectivity
Hardware-in-the-Loop (HWIL) Availability
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca JITC
Redstone (3) DTCC GMAN SED
Charleston (2) IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB
SIMAF
Bethpage NG BAMS
Whiteman B-2
MHPCC
PMRF Bldg 105
Sites in Hawaii
All linked by JMETC
29
Challenges Explained in Detail
30
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
25
FY10 JMETC Events
Customer Event Record Event Dates
Air Force JEFX 10-1Spirit Ice (B2 Data Link Test) October ndash November 2009
Navy BAMS LVC DE October ndash December 2009
Air Force Battlefield Airborne Comm Node (BACN) JUON
(DTOT)
November 2009 to September 2010
Air Force Agile Fire 10-2 January 2010
Air Force JEFX 10-23 FebruaryApril 2010
Joint Joint Surface Warfare JCTD February to September 2010
JS J8JIAMDO Joint Sensor Integration April to September 2010
Air Force B1-B Fully Integrated Data Link Testing April 2010
JFCOM J8489 (TEST WEEK) JCAS Distributed Test June 2010
JIAMDO (Navy Lead) CorrelationDecorrelation Interoperability Test
(CDIT) Integration Events (Continuous)
July to September 2010
Army (Lead) UAS in the National Airspace July to September 2010
Air Force Agile Fire 10-3 August 2010
Joint JITC Joint Interoperability Tests JIT 10-3 amp 11-1
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Joint Strike Fighter (JSF) JIAMDOJoint Track Manager
Brigade Combat Team (BCT)
Modernization
Multi-Function Adv Data Link (MADL) Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run 25
26
JMETC FY10 Accomplishments
JMETC Accomplishments
Supported 88 distinct customer test activities
Expanded network from 38 to 57 sites
ATIN and JTDL Networks transitioned to JMETC
Upgraded JMETC support applications and utilities to TENA R6
DIACAP Tiger Team report completed and recommendations being executed
Enhanced JMETC services and capabilities provided by leveraging InterTEC Services and Industry
Reuse Repository usability improvements
FY10 Example JMETC Customers
Joint Integrated Air amp Missile Defense Organization (JIAMDO)
Broad Area Communications Node (BACN) JUON
B1-B
Broad Area Maritime Surveillance System (BAMS)
Air-Ground Integrated Layer Exploration (AGILE)
Joint Interoperability Test Command (JITC)
Selected Benefits to the DoD
Integrated DT amp OT on a Joint Urgent Operational Need for the warfighter
Maximized usage of theater assets during limited maintenance windows
Improved Joint track information sharing to ensure interoperability of systems in theater operations
Coalition exchange and examination of real-time air picture data
Identification of Air Force Initiatives ready for warfighter transition
Investigated tactical UAS deployment in the National Airspace
Employment of Net-Enabled Weapons
JCAS immediate request amp end-to-end processes ldquoas-isrdquo characterization
Determined distributed system components were not ready for full live integration testing
Executed testing to support system-of-system interoperability certification
27
Customer Event Record Event Dates
Joint JIAMDO CorrelationDecorrelation Interoperability Test (CDIT)
Integration Events (Continuous) OCONUS October 2010
Navy UAS in NAS Runs for Record October 2010
Internal InterTEC Spiral 3 Systems Acceptance Test October - November 2010
Joint JITC Joint Interoperability Tests JIT 11-12345 (Continuous) October 2010 ndash September 2011
Air Force B1-B Fully Integrated Data Link Testing November 2010
Air Force AGILE Fire Phase III JEFX 2011 December 2010 - February 2011
Joint JTRS JPO -- JTRS Ground Mobile Radio January 2011
Navy Broad Area Maritime Surveillance (BAMS) (Continuous) January ndash September 2011
Joint Joint Track Manager Capability Demonstration (Continuous) January ndash September 2011
Joint JSF Initial MampS Interoperability (Continuous) February ndash March 2011
Air Force JSTARS Interoperability Test May 2011
Joint JS J8JIAMDO Joint Sensor Integration June ndash August 2011
FY11 JMETC Events
(More to Come)
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Global Hawk GATOR
Brigade Combat Team (BCT)
Modernization F-22 FY12 Testing Planned Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run
28
JMETC Connectivity
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca (2) JITC JTDL
Redstone (7) SMDC
RTC DTCC GMAN
SED Patriot THAAD FAAD SIV
Charleston (2)IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB SIMAF
As of 16 Mar 2011
0
10
20
30
40
50
60
70
FY06 FY07 FY08 FY09 FY10 FY11
JMETC Site Expansion
Sites
Dedicated trusted connectivity on SDREN (part of the GIG)
Encrypted for Secret ndash System High
DISA-registered IP address space
Active monitoring of network performance
Capable of supporting multiple simultaneous test events Bethpage NGC BAMS
Whiteman B-2
Eglin AFB (4)AOC DTF GWEF KHILS
Hurlburt Field C2DAC
Sites in Gulf Range
Functional Sites 61
New Sites Planned 8
Connection Points to Other Networks 5
Ft Greely CRTC
Site in Alaska
MHPCC
PMRF Bldg 105
Sites in Hawaii
Camp PendletonMCTSSA
China Lake (3)AV-8B FA-18IBAR
Edwards Ridley
Corona NSWC
El Segundo NGC B-2
Sites in SoCal
Point Loma (2)RLBTSSSC-PAC 59140 SSC-PAC HAIPE
Point Mugu (2)ITEC EA-6B
West Agg Rtr
Rancho Bernardo NGC BAMS
Pax River (5)ESTEL E2CD MCLACETEF SAIL ATR JMETC SYSCONEast Agg Rtr
Aberdeen ACCN
Langley (2) C-GIIF TDLITC
JFCOM JSIC
Wallops Island (2) SCSC SSDS
DISA Sky 7
Pentagon WARCAP
Dahlgren (2) CEDLIWSL
Dam Neck CDSA
Newport News NGC VASCIC
Sites in MD DC VA
Norfolk COMOPTEVFOR
McLean MITRE NCEL
Ft Huachuca JTRS
Bethpage NGC BCN
How an Air Force Customer Sees
Distributed Connectivity
Hardware-in-the-Loop (HWIL) Availability
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca JITC
Redstone (3) DTCC GMAN SED
Charleston (2) IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB
SIMAF
Bethpage NG BAMS
Whiteman B-2
MHPCC
PMRF Bldg 105
Sites in Hawaii
All linked by JMETC
29
Challenges Explained in Detail
30
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
26
JMETC FY10 Accomplishments
JMETC Accomplishments
Supported 88 distinct customer test activities
Expanded network from 38 to 57 sites
ATIN and JTDL Networks transitioned to JMETC
Upgraded JMETC support applications and utilities to TENA R6
DIACAP Tiger Team report completed and recommendations being executed
Enhanced JMETC services and capabilities provided by leveraging InterTEC Services and Industry
Reuse Repository usability improvements
FY10 Example JMETC Customers
Joint Integrated Air amp Missile Defense Organization (JIAMDO)
Broad Area Communications Node (BACN) JUON
B1-B
Broad Area Maritime Surveillance System (BAMS)
Air-Ground Integrated Layer Exploration (AGILE)
Joint Interoperability Test Command (JITC)
Selected Benefits to the DoD
Integrated DT amp OT on a Joint Urgent Operational Need for the warfighter
Maximized usage of theater assets during limited maintenance windows
Improved Joint track information sharing to ensure interoperability of systems in theater operations
Coalition exchange and examination of real-time air picture data
Identification of Air Force Initiatives ready for warfighter transition
Investigated tactical UAS deployment in the National Airspace
Employment of Net-Enabled Weapons
JCAS immediate request amp end-to-end processes ldquoas-isrdquo characterization
Determined distributed system components were not ready for full live integration testing
Executed testing to support system-of-system interoperability certification
27
Customer Event Record Event Dates
Joint JIAMDO CorrelationDecorrelation Interoperability Test (CDIT)
Integration Events (Continuous) OCONUS October 2010
Navy UAS in NAS Runs for Record October 2010
Internal InterTEC Spiral 3 Systems Acceptance Test October - November 2010
Joint JITC Joint Interoperability Tests JIT 11-12345 (Continuous) October 2010 ndash September 2011
Air Force B1-B Fully Integrated Data Link Testing November 2010
Air Force AGILE Fire Phase III JEFX 2011 December 2010 - February 2011
Joint JTRS JPO -- JTRS Ground Mobile Radio January 2011
Navy Broad Area Maritime Surveillance (BAMS) (Continuous) January ndash September 2011
Joint Joint Track Manager Capability Demonstration (Continuous) January ndash September 2011
Joint JSF Initial MampS Interoperability (Continuous) February ndash March 2011
Air Force JSTARS Interoperability Test May 2011
Joint JS J8JIAMDO Joint Sensor Integration June ndash August 2011
FY11 JMETC Events
(More to Come)
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Global Hawk GATOR
Brigade Combat Team (BCT)
Modernization F-22 FY12 Testing Planned Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run
28
JMETC Connectivity
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca (2) JITC JTDL
Redstone (7) SMDC
RTC DTCC GMAN
SED Patriot THAAD FAAD SIV
Charleston (2)IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB SIMAF
As of 16 Mar 2011
0
10
20
30
40
50
60
70
FY06 FY07 FY08 FY09 FY10 FY11
JMETC Site Expansion
Sites
Dedicated trusted connectivity on SDREN (part of the GIG)
Encrypted for Secret ndash System High
DISA-registered IP address space
Active monitoring of network performance
Capable of supporting multiple simultaneous test events Bethpage NGC BAMS
Whiteman B-2
Eglin AFB (4)AOC DTF GWEF KHILS
Hurlburt Field C2DAC
Sites in Gulf Range
Functional Sites 61
New Sites Planned 8
Connection Points to Other Networks 5
Ft Greely CRTC
Site in Alaska
MHPCC
PMRF Bldg 105
Sites in Hawaii
Camp PendletonMCTSSA
China Lake (3)AV-8B FA-18IBAR
Edwards Ridley
Corona NSWC
El Segundo NGC B-2
Sites in SoCal
Point Loma (2)RLBTSSSC-PAC 59140 SSC-PAC HAIPE
Point Mugu (2)ITEC EA-6B
West Agg Rtr
Rancho Bernardo NGC BAMS
Pax River (5)ESTEL E2CD MCLACETEF SAIL ATR JMETC SYSCONEast Agg Rtr
Aberdeen ACCN
Langley (2) C-GIIF TDLITC
JFCOM JSIC
Wallops Island (2) SCSC SSDS
DISA Sky 7
Pentagon WARCAP
Dahlgren (2) CEDLIWSL
Dam Neck CDSA
Newport News NGC VASCIC
Sites in MD DC VA
Norfolk COMOPTEVFOR
McLean MITRE NCEL
Ft Huachuca JTRS
Bethpage NGC BCN
How an Air Force Customer Sees
Distributed Connectivity
Hardware-in-the-Loop (HWIL) Availability
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca JITC
Redstone (3) DTCC GMAN SED
Charleston (2) IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB
SIMAF
Bethpage NG BAMS
Whiteman B-2
MHPCC
PMRF Bldg 105
Sites in Hawaii
All linked by JMETC
29
Challenges Explained in Detail
30
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
27
Customer Event Record Event Dates
Joint JIAMDO CorrelationDecorrelation Interoperability Test (CDIT)
Integration Events (Continuous) OCONUS October 2010
Navy UAS in NAS Runs for Record October 2010
Internal InterTEC Spiral 3 Systems Acceptance Test October - November 2010
Joint JITC Joint Interoperability Tests JIT 11-12345 (Continuous) October 2010 ndash September 2011
Air Force B1-B Fully Integrated Data Link Testing November 2010
Air Force AGILE Fire Phase III JEFX 2011 December 2010 - February 2011
Joint JTRS JPO -- JTRS Ground Mobile Radio January 2011
Navy Broad Area Maritime Surveillance (BAMS) (Continuous) January ndash September 2011
Joint Joint Track Manager Capability Demonstration (Continuous) January ndash September 2011
Joint JSF Initial MampS Interoperability (Continuous) February ndash March 2011
Air Force JSTARS Interoperability Test May 2011
Joint JS J8JIAMDO Joint Sensor Integration June ndash August 2011
FY11 JMETC Events
(More to Come)
Discussions for Future Teaming
Gerald R Ford Class (CVN-78) Global Hawk GATOR
Brigade Combat Team (BCT)
Modernization F-22 FY12 Testing Planned Multi-Mission Maritime Aircraft (MMA)
Each event is normally preceded by 1-3 spirals Connectivity Check Integration and Dry Run
28
JMETC Connectivity
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca (2) JITC JTDL
Redstone (7) SMDC
RTC DTCC GMAN
SED Patriot THAAD FAAD SIV
Charleston (2)IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB SIMAF
As of 16 Mar 2011
0
10
20
30
40
50
60
70
FY06 FY07 FY08 FY09 FY10 FY11
JMETC Site Expansion
Sites
Dedicated trusted connectivity on SDREN (part of the GIG)
Encrypted for Secret ndash System High
DISA-registered IP address space
Active monitoring of network performance
Capable of supporting multiple simultaneous test events Bethpage NGC BAMS
Whiteman B-2
Eglin AFB (4)AOC DTF GWEF KHILS
Hurlburt Field C2DAC
Sites in Gulf Range
Functional Sites 61
New Sites Planned 8
Connection Points to Other Networks 5
Ft Greely CRTC
Site in Alaska
MHPCC
PMRF Bldg 105
Sites in Hawaii
Camp PendletonMCTSSA
China Lake (3)AV-8B FA-18IBAR
Edwards Ridley
Corona NSWC
El Segundo NGC B-2
Sites in SoCal
Point Loma (2)RLBTSSSC-PAC 59140 SSC-PAC HAIPE
Point Mugu (2)ITEC EA-6B
West Agg Rtr
Rancho Bernardo NGC BAMS
Pax River (5)ESTEL E2CD MCLACETEF SAIL ATR JMETC SYSCONEast Agg Rtr
Aberdeen ACCN
Langley (2) C-GIIF TDLITC
JFCOM JSIC
Wallops Island (2) SCSC SSDS
DISA Sky 7
Pentagon WARCAP
Dahlgren (2) CEDLIWSL
Dam Neck CDSA
Newport News NGC VASCIC
Sites in MD DC VA
Norfolk COMOPTEVFOR
McLean MITRE NCEL
Ft Huachuca JTRS
Bethpage NGC BCN
How an Air Force Customer Sees
Distributed Connectivity
Hardware-in-the-Loop (HWIL) Availability
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca JITC
Redstone (3) DTCC GMAN SED
Charleston (2) IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB
SIMAF
Bethpage NG BAMS
Whiteman B-2
MHPCC
PMRF Bldg 105
Sites in Hawaii
All linked by JMETC
29
Challenges Explained in Detail
30
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
28
JMETC Connectivity
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca (2) JITC JTDL
Redstone (7) SMDC
RTC DTCC GMAN
SED Patriot THAAD FAAD SIV
Charleston (2)IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB SIMAF
As of 16 Mar 2011
0
10
20
30
40
50
60
70
FY06 FY07 FY08 FY09 FY10 FY11
JMETC Site Expansion
Sites
Dedicated trusted connectivity on SDREN (part of the GIG)
Encrypted for Secret ndash System High
DISA-registered IP address space
Active monitoring of network performance
Capable of supporting multiple simultaneous test events Bethpage NGC BAMS
Whiteman B-2
Eglin AFB (4)AOC DTF GWEF KHILS
Hurlburt Field C2DAC
Sites in Gulf Range
Functional Sites 61
New Sites Planned 8
Connection Points to Other Networks 5
Ft Greely CRTC
Site in Alaska
MHPCC
PMRF Bldg 105
Sites in Hawaii
Camp PendletonMCTSSA
China Lake (3)AV-8B FA-18IBAR
Edwards Ridley
Corona NSWC
El Segundo NGC B-2
Sites in SoCal
Point Loma (2)RLBTSSSC-PAC 59140 SSC-PAC HAIPE
Point Mugu (2)ITEC EA-6B
West Agg Rtr
Rancho Bernardo NGC BAMS
Pax River (5)ESTEL E2CD MCLACETEF SAIL ATR JMETC SYSCONEast Agg Rtr
Aberdeen ACCN
Langley (2) C-GIIF TDLITC
JFCOM JSIC
Wallops Island (2) SCSC SSDS
DISA Sky 7
Pentagon WARCAP
Dahlgren (2) CEDLIWSL
Dam Neck CDSA
Newport News NGC VASCIC
Sites in MD DC VA
Norfolk COMOPTEVFOR
McLean MITRE NCEL
Ft Huachuca JTRS
Bethpage NGC BCN
How an Air Force Customer Sees
Distributed Connectivity
Hardware-in-the-Loop (HWIL) Availability
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca JITC
Redstone (3) DTCC GMAN SED
Charleston (2) IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB
SIMAF
Bethpage NG BAMS
Whiteman B-2
MHPCC
PMRF Bldg 105
Sites in Hawaii
All linked by JMETC
29
Challenges Explained in Detail
30
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
How an Air Force Customer Sees
Distributed Connectivity
Hardware-in-the-Loop (HWIL) Availability
Army
Air Force
Navy
Marines
Joint
Industry
Ft Huachuca JITC
Redstone (3) DTCC GMAN SED
Charleston (2) IPC MEF-MEU
Ft Hood (2) CTSF TTEC
WPAFB
SIMAF
Bethpage NG BAMS
Whiteman B-2
MHPCC
PMRF Bldg 105
Sites in Hawaii
All linked by JMETC
29
Challenges Explained in Detail
30
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
Challenges Explained in Detail
30
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
Mission Threads and Measures
bull JFCOM J89 leading work on Mission Threads
o Tier 1 Working ndash EWEA Joint Fires TST Integrated Tactical
Warning
o Tier 1 Complete ndash Counter IED and Global Force Management
o Tier 1 2 3 Working ndash JCAS JPR Air amp Missile Defense
bull Joint Test amp Evaluation Methodology amp JTEM Transition
have led Mission Measures work
o Measures Development Standard Operating Procedure (SOP) to
DOTampE (Sep 10 and Jan 11)
o Mission-Based Test amp Evaluation Assessment Process Guidebook
to DOTampE (Mar 11)
o httpsaccdaumiltije (request an account at
httpsaccdauCommunityBrowseraspx)
bull Much work has been done
bull Much work is yet to do 31
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
Information Assurance
bull DIACAP Tiger Team
o Brought the RDTampE community case to the authors of DIACAP
o JMETC is now a non-voting member of the DIACAP Technical
Advisory Group
o Look for revisions to DIACAP in the near future
o NOTE Ryan I am hoping I can set someone up for this on your
panel
bull Much work to be done
o Streamlined but thorough processes
o Automate the processes where feasible
o Automate management of ports and protocols
32
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
Multi-Level Security
bull Three cases
o Classification
o Coalition
o Proprietary data
bull Need an enterprise-level solution
o Reduce approval times
o Pre-defined solutions
bull CTEIP Multi-Level Secure (MLS) JointCoalition Network
Environment project is in requirements determination
bull Personally I would like the solution to be tied to TENA but
I am not driving the requirements
33
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
Double Encryption
bull Many tactical systems have their own encryption for data
o JTRS
o BAMS
o Others
bull SDREN encrypts all data traversing the network
bull If data is encrypted twice what is the effect
o Are there data issues with double encryption
o Is there a packet loss problem with double encryption
o Is there a work-around to avoid double encryption
o Is there a solution for double encryption
34
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
Near Real Time Test-Fix-Test
bull Requires significant planning
o Test objectives ndash what questions are we trying to answer
o Data collection ndash what data do we need to answer those questions
o Test execution ndash SUT scenarios other systems (blue amp red)
environment etc
o Data management ndash TRMC working on requirements
o Data reduction
o Analysis
bull JIAMDO is very close
o Frank Barone helping to understand the possibilities
o Running a second shift to accommodate
o IDrsquoing issues one day and testing fixes the next
35
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
Data Management
bull Analyst should call for the data heshe needs
o Without regard to where it is stored
o Without regard to the format
Wersquove been trying to standardize data formats between Services and
ranges for 30 years ndash without success
bull And receive that data within minutes
bull Marty Arnwine leading an effort to determine community
requirements
o Requirements will be submitted to the CTEIP for consideration
bull Wersquoll never get where we need to be without very efficient
data management
36
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
Imbedded amp Unobtrusive
Instrumentation
bull If we want to provide information to the Warfighter
o We better measure performance of his systems
o That requires that we have instrumentation on his systems
Unobtrusive
Imbedded
Data transmitted wirelessly
bull TRMCrsquos TampESampT Program
o Testing in a Resource Constrained Environment
Demonstration here
o Imbedded Instrumentation
bull CTEIP
o Integrated Network Enhanced Telemetry
o InterTEC
o Net-Centric Test Agent Capability
37
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38
Frequency Spectrum
bull Testing for the Warfighter implies world-wide data
collection
bull It also implies wireless data collection since we cannot
interfere with combat operations
bull This implies extremely efficient use of the frequency
spectrum
bull My boss Mr Hinton is next-up to address this issue
38