FORMULATION AND IMPLEMENTATION OF A METHOD FOR … · FORMULATION AND IMPLEMENTATION OF A METHOD FOR TECHNOLOGY EVALUATION OF SYSTEM OF SYSTEMS Fig. 2 Technology Identiﬁcation,

FORMULATION AND IMPLEMENTATION OF A METHOD FORTECHNOLOGY EVALUATION OF SYSTEM OF SYSTEMS

Dimitri N. Mavris∗, Alicia Sudol∗∗Georgia Institute of Technology

Keywords: System of Systems, Operational Analysis, Technology Evaluation, Discrete EventSimulation

Abstract

In a fiscally constrained environment, invest-ments in technologies for military systems of sys-tems (SoS) must yield improvements in opera-tional effectiveness as well as individual systemor vehicle performance. The current technologyassessment approaches, which focus on system-level impacts, could be enhanced by expandingthe design space to a SoS perspective, leadingto improved cost/benefit analyses. The method-ology presented herein, provides a capability totrade and compare operational benefits of tech-nological upgrades by quantifying the impactsfrom an SoS perspective. The resulting designspace is expanded beyond individual vehicle siz-ing and performance parameters to include mul-tiple system performance parameters and the re-sulting measures of effectivness from a missionsimulation. A proof-of-concept is demonstratedbased on an Australian Humanitarian Aid andDisaster Relief Mission during Cyclone Pam in2015. The approach is able to identify the impactof advanced vehicle technologies on overall SoSmission-level effectiveness.

1 Introduction

The increasing advancement of information ex-change promotes capabilities based on collabo-ration between independent systems, requiringmodern military systems to be defined and evalu-ated as interconnected Systems of Systems (SoS)[1]. An SoS architecture in a military environ-

ment may be comprised of a collection of missionstrategies and/or collections of ground vehicles,aircraft, surface ships, and satellites. The newsystems being developed today will be interoper-able, and related technologies must be designedfrom an SoS perspective with situational aware-ness of the intended operating environment, re-quiring a change in the engineering mindsest.

Typically, most technology evaluations con-sist of a design space exploration performed atthe system level for a fixed mission around abaseline vehicle configuration. In other words,the means to perform this mission are variedwhile the ways in which the mission is completedare fixed [2]. In a similarly limited process, op-erational analysis typically involves selecting afixed vehicle or set of vehicles to investigate al-ternative tactics to complete a given mission. Inthis example, the means to perform the missionare fixed while the ways the mission is performedare varied. This disjointed process results in op-erational inefficiencies as advances in technologyincrease the need for complex system of systems[3].

The methodology discussed herein allows theimpact of new technologies to be assessed fromoperational measures of effectiveness at the SoS-level by expanding the design space from individ-ual vehicle sizing and performance parameters toinclude multiple system sizing and performanceparameters as well as potential operating condi-tions, or alternative tactics. By deriving require-ments from the top-level for new systems, an im-provement in overall future fleet development can

1

DIMITRI N. MAVRIS, ALICIA SUDOL

be achieved beyond isolated vehicle performanceimprovements.

The foundation of this approach is based onthe Technology Identification, Evaluation, andSelection (TIES) method developed, initially, forcommercial transport aircraft [2]. This vehicletechnology assessment approach has been mod-ified to include the impact of individual vehi-cle technologies on Measures of Performance(MoPs), or means, and the impact of complex in-teractions among multiple elements on mission-level Measures of Effectiveness (MoEs), or ways.With a focus on military transport aircraft, aproof-of-concept example is shown based on anAustralian Humanitarian Aid and Disaster ReliefMission (HADR) during Cyclone Pam in 2015.The approach is able to identify the impact ofadvanced vehicle technologies on overall SoSmission-level effectiveness.

2 Method Description

To aid in the methodology development, thegeneric decision-making process, shown in Fig.1, is used as a guide [4]. This process breaksdown the fundamental steps of a generic problemand logically constructs the development of a so-lution. These steps include Establish the Need,Define the Problem, Establish Value, GenerateFeasible Alternatives, Evaluate Alternatives, andMake Decision. The need for this methodology

Fig. 1 Generic decision-making process flowchart [4].

was discussed in the introduction, where the needfor an SoS-level perspective in vehicle technol-ogy assessment was identified based on the in-creasingly complex and interconnected systemsbeing utilized for military operations. In defin-ing the problem, the challenges in solving it areidentified and addressed.

The largest challenge is the vastly increaseddesign space for SoS analysis, which includes in-numerable potential combintations and trades be-tween means and ways. To enable these trades tobe performed rapidly, key enablers are required.Some of these enablers include integrated analy-sis models, probabilistic analysis, multi-objectiveand multi-attribute decision making techniques,creation of a parametric tradeoff environment,and the use of surrogate models. These methodshelp meet the desire to take a complicated, com-binatorial design problem and allow for a thor-ough examination of the available options in atimely manner.

Another key enabler is the existing TIESmethodology, discussed in Ref. [2]. The TIESsteps are shown graphically in Fig. 2 and re-peated below:

1. Problem definition

2. Baseline and alternative concept identifica-tion

3. Modeling and simulation

4. Design space exploration

5. Determination of system feasibility and vi-ability

6. Technology identification

7. Technology evaluation

8. Technology selection

TIES introduces a structured approach to quicklyand accurately identify technically feasible andeconomically viable alternatives. While this pro-cess is effective for identifying combinations oftechnologies that meet both performance and costrequirements for a single vehicle, it currently

2

FORMULATION AND IMPLEMENTATION OF A METHOD FOR TECHNOLOGY EVALUATION OFSYSTEM OF SYSTEMS

Fig. 2 Technology Identification, Evaluation, and Selection (TIES) [2].

does not consider the impact of multiple assetsfor a given mission nor does it evaluate alterna-tive tactics for these systems. During the mod-eling and simulation step of the TIES process, abaseline mission is selected to define the perfor-mance requirements on the vehicle being mod-eled. The vehicle modifications due to technol-ogy integration are modeled as changes in thetechnical metrics that either improve or degradethe vehicle performance during this baseline mis-sion. To overcome these limitations, an expandedand iterative approach to TIES has been devel-oped by the authors to directly address the issuesthat impact a system of systems.

Fig. 3 introduces the extension of TIES to theSoS problem, TIES-SoS. The steps in TIES-SoSinclude the use of SoS modeling and simulationtechniques to derive vehicle requirements and aniteration of TIES over multiple vehicle and tech-nology alternatives. The entire process is theniterated until peak SoS performance is identifiedthrough means vs. ways design space evaluationand tradeoffs.

The next steps in the IPPD guideline coin-cide with steps in the TIES process. To establishthe value of means and ways trades, the vehiclesand technology alternatives must be abstractedand defined by MoPs relating to these vehicles,while the simulation of alternative tactics mustbe abstracted and defined by MoEs. Generat-ing and evaluating means and ways alternativesis an expansion of the TIES technology identi-fication and evaluation steps to include tacticaldeviations and assessing their operational impactwhen combined with technology enhancements.

Finally, the Make Decision step involves select-ing technologies or combinations of technologiesfor the TIES process and selecting from alterna-tive strategies that include combinations of tac-tics and technologies for an SoS. These steps arefurther discussed in the following section, whereTIES-SoS is introduced.

2.1 TIES-SoS

Define The Problem: The first step in the TIESmethodology is problem definition. This step re-quires a translation of qualitative customer re-quirements to quantifiable and testable require-ments. Management and planning tools like theQuality Functional Deployment (QFD) are oftenused in this phase. QFD is a mapping of customerattributes to engineering characteristics. System-level MoPs and mission-level MoEs are identifiedduring this initial step.

Define Concept Space: The conceptual de-sign space for each system includes all optionsfor the various subsystems that satisfy the func-tional requirements. From a system of systemsperspective, this also includes options for the sys-tem interactions. For a military campaign, theseoptions could include the number of ground or airvehicles as well the tactics that relate to these op-tions. How are these vehicles/systems going to beoperated for a given mission or set of missions?What are the alternative Concepts of Operations(CONOPS)? Do those options change for differ-ent system of systems architecture? A baselinevehicle design must be established for every sys-tem option in the SoS design space, and a base-

3


Fig. 3 Technology Identification, Evaluation, and Selection for System of Systems (TIES-SoS)

line set of operational parameters must be estab-lished for those vehicles. The options are enu-merated in a Morphological Matrix to aid design-ers in identifying potential combinations of sub-systems. A baseline mission and vehicle designis established in this step as a datum point to startthe design space exploration. Variations on thisbaseline are generated by selecting other optionsin the morphological matrix and are consideredalternative vehicle concepts.

Modeling and Simulation: A modeling andsimulation environment is used to assess the tech-nical feasibility of the design alternatives andevaluate their system metrics. For systems, theseenvironments can be regressions based on his-torical data, detailed physics-based analysis, oranything in between. SoS modeling and sim-ulation environments include varying levels ofmission abstraction. Typical approaches includesystem dynamics, discrete event simulation, oragent based models. These techniques will bediscussed in more detail later.

Investigate Design Space: The design spaceexploration begins by determining datum values

for the metrics of interest via alternative con-cept modeling in the M&S environment. Thedesign space for a baseline, or conventional, de-sign is initially investigated by varying the designparameters. There are three probabilistic meth-ods to explore the design space and identify fea-sible/viable solutions which provide cumulativedistribution functions (CDF) for each metric:

• Link simulation code with Monte Carlosimulation

• Create a Metamodel and link to MonteCarlo model

• Monte Carlo simulation methods

Surrogate Models: If the existing designspace does not contain feasible viable solutions,the design space can be increased by either re-laxing the constraints, implementing alternativetactics, or infusing technologies. The impactof technologies is determined using technologymetric k-factors. These k-factors modify disci-plinary technical metrics that are calculated us-ing the sizing and synthesis tool. This modifica-tion represents the benefit or penalty associated

4


with infusing a new technology. Similarly, theimpact of alternative tactics is determined usinganalogous λ-factors which modify operational ef-fectiveness metrics calculated using SoS scenariosimulations. Due to the significant increase ispossible alternatives between technology and tac-tic combinations, surrogate modeling techniquesare utilized to approximate the analysis tools. Re-sponse Surface Equations and Artificial NeuralNetworks are two frequent approaches to developsurrogate models.

Identify Means and Ways Alternatives: Ifno solutions are found in the design space inves-tigation, potential technologies for infusion andadditional CONOPS that effectively utilize thosetechnologies must be identified. The k-factor andλ-factor projections for these alternatives requirephysical compatibilities and quantitative impactsto be determined. A compatibility matrix is usedto formalize the technology and tactic compati-bilities in a pairwise comparison.

Evaluate Means and Ways Alternatives:A Technology Impact Matrix (TIM) defines theprojected quantitative impacts of each technol-ogy on the system metrics. The ‘k’ factorsmodify the technical metrics used during anal-ysis, such as range or specific fuel consump-tion, simulating technology benefits and penal-ties. The technology infusion is categorical innature, where the ordering of the tactics and tech-nologies in the impact matrix does not matter. Anotional TIM is shown in Fig. 4. A similar matrixis created to evaluate the impacts of tactical alter-natives, represented by vectors of λ-factors. Anotional tactic impact matrix is shown in Fig. 5.The impact metrics can be added/removed inde-pendently of the others, assuming compatibilitiesare met, and can be treated as additive during theanalysis.

To create a design of experiments that en-ables a strategic investigation of the tactic andtechnology tradespace, the number of selectionsfor each alternative is equal to the number oftechnologies plus the number of tactics, n + m.This design space yields 2n+m combinations, as-suming all combinations are possible, and cre-ates a large combinatorial problem. For exam-

Fig. 4 Notional technology impact matrix.

Fig. 5 Notional tactic impact matrix.

ple, if there were 10 technologies and 30 tacticsto consider, the design space would contain 240,approximately 1.1 trillion combinations. A no-tional design of experiments is shown in Fig. 6.Additionally, the technology and tactic impactsare probabilistic, implying the need to generate aCDF for each combination.

Fig. 6 Notional design of experiments for tech-nology and tactical combinations.

The multi-level modeling and simulation en-vironment that includes both system-level sizingmodels and SoS-level mission analysis are in-tegrated using the surrogate models and ‘k’/‘λ’factor metric modifiers into a multi-level Uni-fied Tradeoff Environment (UTE) [5]. The UTE,which is shown in the surrogate modeling blockof Fig. 3 and is blown up in Fig. 7, allows simul-taneous trades to be performed between designvariables, requirements, tactics or technologiesat each level of the hierarchy. By utilizing sur-

5


rogate models, a decision-maker can adjust eachcomponent and see the impact cascade throughthe multi-level analysis immediately for real-timesensitivity analysis and trades. Powerful graphi-cal tools like the UTE provide insight into eachlayer of the problem simultaneously.

Fig. 7 Hierarchical integrated Unified TradeoffEnvironment.

Strategy Selection: Due to the multi-attribute, multi-objective nature of these complexdesigns, various techniques can be used to se-lect means and ways combinations. Examples in-clude direct resource allocation through 1-1 com-parisons, scoring models such as the Techniquefor Order Preference by Similarity to Ideal So-lutions (TOPSIS), or frontiers that compare theeffectiveness of alternatives technologies and tac-tics. A notional frontier plot is shown in Fig. 8.

Fig. 8 Notional technology frontier comparingtechnology effectiveness and investment require-ments.

For more immediate effectiveness gains, al-ternative ways are potentially the quickest change

to implement. Investments in technology devel-opment, or combinations of technologies, pro-vides the opportunity for further growth. Whilethe combination of technologies and tactics al-lows for the most improvement, where the tech-nologies can be fully exploited through an idealset of tactics.

3 System of Systems Modeling and Simula-tion Approaches

One of the more significant deviations from TIESis the alternative approaches to system of sys-tem modeling and simulation as compared to sys-tems modeling, which range in their level of ab-straction. For example, manufacturing simula-tion can range from detailed schedules, latencies,and capacity of individual processes, while sup-ply chain management may not deal with indi-vidual packets and use volumes instead. The ma-jor approaches include dynamic systems, discreteevent simulation, and agent-based models and areshown on an abstraction scale in Fig. 9 [6].

Dynamic systems are systems that are notstatic, i.e. their state changes over time. The un-derlying mathematical model consists of physics-based equations or relationships defined throughexperimentation. This type of modeling is oftenused in engineering disciplines, such as mechani-cal, electrical, chemical, etc. [6]. Graphical mod-eling languages can be used to design control sys-tems using this technique.

Discrete Event Simulation (DES) is a tech-nique which models the event-based behavior ofa system. Using mathematical or logical mod-els of the physical system, DES portrays statechanges at precise points in simulated time [7, 8].A DES is comprised of active entities, where ac-tions are performed as part of the system model,and passive entities, which include time delaysor queues and resource utilization. DES is fre-quently used to model the real-life behavior of afacility or system, including manufacturing pro-cess flows and military logistics [9]. For theseproblems, the activity-based approach to DES al-lows the vehicle abstraction required of this so-lution, while minimizing the behavioral uncer-

6


Fig. 9 Approaches in Simulation Modeling on Abstraction Scale [6].

tainty. It models the resource limitation chal-lenge that is key to any logistics problem, andcan include stochasticity of random, unscheduledevents.

Agent-based modeling (ABM) is a morecomplex modeling technique commonly used infields ranging from ecology, game theory, andSoS design [6]. These models are created bydefining the simple behaviors of individual, low-level agents through a set of rules, and then initi-ating the simulation to observe how these agentsinteract [6, 10]. This type of simulation is de-signed to identify and explain emergent behav-iors that result when many systems interact [11].The primary characteristics for these multi-agentsystems include the restriction of information andcapabilities to each agent, the distributed sys-tem control, decentralization of data, and asyn-chronous computation [12]. This modeling tech-nique is advantageous for military engagementmodeling, where the logic of individual systems(soldiers, ground vehicles, autonomous vehicleswarms, etc) are easier to define than their com-plex interactions. While this technique requiresmore complex modeling, the valuation of com-

plex interactions can provide critical informationfor military mission analysis [13].

4 Proof of Concept Implementation

The TIES-SoS approach is demonstrated on ahumanitarian aid/disaster relief (HADR) missionoccurring in the geographically dispersed islandsof Fiji, based on the role of the Australian mili-tary during Cyclone Pam in 2015. In this appli-cation example, the goal is to inform the require-ments of modern military acquisition processesby examining the mission effectiveness of Verti-cal Takeoff and Landing (VTOL) asset acquisi-tion alternatives. The overall technical approachutilized for this proof-of-concept is shown in Fig.10. In this graphical representation, the TIES-SoS steps can be seen throughout the imple-mentation, including defining the baseline mis-sion by selecting a CONOPS, identifying meansand ways alternatives by researching relevanttechnologies and possible operational methods,among others. A full description of the frame-work developed in this example and the detailedresults can be found in Ref. [14].

7


Fig. 10 Graphical representation of technical ap-proach for the modeling and simulation environ-ment.

4.1 Problem Definition

Experience with cyclones Pam and Winston hasshown that the first five days of relief efforts arethe critical window to maximize efforts of verti-cal lift assets before naval assets arrive with addi-tional capabilities and support. However, some ofthe five days are consumed by transporting verti-cal lift assets to the area of operations (AO) dueto the distance. Maximizing sortie generation ofdirect-support vertical lift assets in the AO in thefive-day window is a primary metric, which isanalogous to maximizing the number of relief-packages delivered. Limiting support personnel,minimizing vehicle downtime, and quickly iden-tifying priority areas for relief are other criticalfactors affecting mission success. These criticalfactors are then translated to MoPs for the verti-cal lift assets and MoEs for the HADR mission.

4.2 Define Concept Space

The morphological matrix shown in Fig. 11 enu-merates the vehicle architecture and tactical op-tions for the HADR mission. The vehicle archi-tecture options must be selected for each typeof vehicle that is operating in the SoS, includ-ing the number available, the range and capacityof that vehicle, transit frequency, alternative fea-tures, and capability in extreme weather. The tac-tical options for these architectures include pack-age delivery methods, refueling capability andoperability, and degree of autonomous operationsfor any autonomous vehicles considered. Oper-

ational MOPs for the five-day operation are thetime available for mission, down-time of vehi-cles, number of sorties generated in theater, andnumber of hours flown by vehicles. The Oper-ational MOEs for the five-day operation are thenumber of packages delivered and the number ofTikinas (analogous to counties) visited. TheseMOEs translate directly to percent of people ser-viced and percent of country serviced, respec-tively.

4.3 Baseline Mission/Asset Definition andAlternative Concept Identification

The phases of a disaster relief operation aredefined by different goals and sets of ac-tions, where each can be modeled individually.Phases 0 and I, ‘Receipt of Mission’ and ‘Pre-deployment/Staging Operations’ respectively, areawareness and prepping stages. Phase 0 includesprioritizing goals, identifying constraints, deter-mining resources required and resources avail-able, and determining the availability of units fordeployment. Phase I includes mobilizing, prepar-ing personnel and equipment for transportation,and loading the strategic airlift. For this proofof concept, Phases 0 and I are not included inthe simulation. It is assumed that the availableHADR assets are already prepped in the stagingarea.

The five day window for operations in thissimulation begins with Phase II, ‘Deployment’.Deployment options for relief assets include ei-ther self-deployment or transportation via C-17to the AO. The simulation time clock begins atthis stage, and the transportation time to the AOis determined based on method of deployment.Transit time for assets that are strategically air-lifted is equivalent to the transit time of the strate-gic airlift (STRATAIR). The self-deployed unitsrequire additional resources and time for refuel-ing and stopovers as necessary. Phase III, ‘Re-ception, Staging, Onward movement, and Inte-gration (RSOI)’ consists of support personneland equipment arrival and preparation for reliefoperations. Staging for all assets is assumed to bein place and not accounted for in the model. The

8


Fig. 11 Morphological Matrix enumerating vehicle and tactical options for the HADR mission.

STRATAIR deployment method requires the as-sets to be reassembled and to undergo test flightsbefore they are considered operational, consum-ing time in the simulation. Phase IV, ‘ConductOperations’ is all of the relief mission logis-tics and operations, including distribution of aidpackages, medical evacuations, or any other nec-essary support. Phase V is ‘Redeploy’, which oc-curs after the initial assigned HADR operationsare complete, and is therefore, not considered inthis simulation.

To define the baseline vehicle assets, a re-view of the existing Australian Army mediumand heavy-lift helicopters was conducted. Thesevehicles include the CH-47F (Chinook), MRH-90, and S-70A-9 (Black Hawk). To define thesebaseline vehicles for the modeling and simulationenvironment, the performance parameters, geo-metric constraints, and operating conditions foreach of these vehicles was identified. These base-line values are used in the modeling and simu-lation environment to determine the throughputcapabilities of baseline SoS architecture combi-nations.

4.4 Investigate Design Space

Investigating the design space entails identifyingany scenario requirements and constraints andenumerating the existing options that potentiallymeet those requirements. Often this explorationleads to the realization that there are few to nofeasible existing options, driving the need forevaluating alternative CONOPS and/or infusingnew technologies. The Fiji scenario reduces thedesign space based on local geography and re-sources. Access to available airports or air basesfor STRATAIR and VTOL assets, environmen-tal considerations including limited visibility andpresence of wind gusts due to rough weather, thestate of existing local communication structures,etc. are all potential factors in this mission sce-nario. For the proof-of-concept problem, only 2C-17s are available for relief operations and arelimited to two sorties. The local command andcontrol center is assumed to have no communi-cations with outlying islands, and a distributionmethod was implemented based on existing pop-ulation statistics. Based on the baseline vehi-cle performance parameters, only the CH-47 andMRH-90 are able to reach the outermost Fiji is-lands from the Royal Australian Air Force base

9


in Amberley.

4.5 Modeling and Simulation

Discrete Event Simulation, which is commonlyused for military logistics problems, was chosenfor this problem. The structure of the DES is ca-pable of handling parallel tasks and events, butrequires a new simulation to be run for new ve-hicles or changes in the operational procedures.To speed up the run time and reduce computa-tional resources required, surrogate models wereused to capture the impact of vehicle and missionchanges on the MoPs and MoEs.

4.6 Surrogate Modeling

A design of experiments (DOE) was createdaround the discretized operation and performancemetrics used to define the baseline vehicle andmission scenarios within the simulation environ-ment. Separate DOEs for each asset type werecreated due to the categorical nature of the prob-lem, with variations on the performance metricsof a fixed percentage from the baseline. NeuralNetworks were used to create a surrogate modelfrom the resulting DOE data, and the surrogatemodel fits were evaluated based on Ref. [15].The considered assets did not cover the full rangefor each performance metric aggregated acrossvehicle types using this approach. While the sur-rogate models performed better by limiting thebaseline variations, there were gaps in capabilityfor potential alternatives.

4.7 Identify Means and Ways Alternatives

To generate the means and ways alternatives, themission operating parameters and vehicle perfor-mance parameters are decomposed as shown inTable 1 [14]. The operating parameters representtactical options or ways, while the vehicle param-eters represent different technologies or means.

Means: Technology improvements were di-vided into three different types: general air-frame technologies, reduced/zero maintenancetechnologies, and engine core technologies. Gen-eral airframe technologies encompass potential

technologies that could be developed to reducedrag, rotor tip losses, aircraft weight, or the like.This may include technologies such as swept ro-tor tips or composite landing gear. Reduced/ZeroMaintenance technologies reflect technologiescurrently under development which may be ap-plied to multiple systems in the aircraft, mostnotably within the powertrain and engines, dra-matically reducing the amount of maintenancerequired both in terms of scheduled downtimeas well as mean time between failure. Enginecore technologies improve engine core cycle per-formance, particularly in the specific fuel con-sumption. Possible technologies include com-pressor blisks and thermal barrier coated turbineblades. The vehicle parameters affected by thesetechnologies were translated to ‘k’-factors thatvary as a percent change from baseline values foreach aircraft considered. Empty weight, max-imum take-off weight, and cruise specific fuelconsumption were varied up to ±15% from base-line, and all other parameters were varied up to±25% from baseline.

Ways: Transport to the AO for these assetscan be accomplished by either self-deploymentor strategic airlift via cargo transport. Self-deployment requires refueling options for verti-cal airlift vehicles that cannot fly to the destina-tion in one tank of fuel. Options for refueling in-clude aerial refuel or gas-go, which requires land-ing at a waypoint en-route to the destination. De-pending on the time required and crew rest limi-tations, the asset can simply refuel or may be re-quired to remain over night. Strategic airlift elim-inates the need for refueling transporting the as-set within a cargo aircraft in a direct flight, butrequires the assets to be reassembled upon arrivaland to undergo test flights before they are consid-ered operational. The deployment method sig-nificantly impacts the total time available for themission, and should be minimized where possi-ble.

Autonomous operations primarily relaxedconstraints on crew requirements for each vehi-cle. HADR missions are sensitive to the num-ber of personnel on site. As the number of per-sonnel increases, the required infrastructure over-

10


Table 1 Potential technologies and tactics for HADR mission.Parameter Group Alternative Aircraft Parameter

Vehicle Parameters

General Airframe Technologies

Maximum Takeoff WeightEmpty Weight

Maximum Fuel AvailableCruise Speed

Reduced/Zero Maintenance TechnologiesMean Time Between FailureMean Scheduled Downtime

Engine Core TechnologiesCruise Specific Fuel Consumption

Combat Radius

Operating Parameters

Deployment Alternatives Aerial Refuel

Autonomy AlternativesAutonomous Operations

Semi-Autonomous OperationsTeleoperations

head to support the mission will also increase.Autonomy potentially allows the amount of per-sonnel on site to be reduced while retaining mis-sion effectiveness. While other effects may exist,these were not considered as part of this study.The levels of autonomy were mapped to differentnumbers of crew per vehicle. Fully autonomoussystems assumed zero crews per vehicle, semi-autonomous and teleoperated systems assumed0.5 crews per vehicle. Conventional systems re-tained the one crew or more per vehicle require-ment. In this context, ‘crews’ refer to pilots andoperating crewmembers exclusively. These al-ternative operating parameters are mapped to λ-factors affecting the simulation.

4.8 Evaluate Means and Ways Alternativesand Select Strategies

In this example, a parametric dashboard was cre-ated to allow users to interactively compare ac-quisition alternatives for existing and technology-infused assets under different operating con-ditions. The multi-attribute decision-making(MADM) tool is shown in Fig. 12 and consistsof three primary sections: location selection, in-put variables, and output graphs. The inputs in-clude asset selection and its associated deploy-ment method, crew availability, and loading type.These selections are used to define the baselinevalues for all aircraft parameters.

The MoP and MoE analysis results are allplotted in the output section, which is dividedinto three different categories: mission details,mission results, and mission resources. The out-puts include time available for operation, flighthours, and maintenance hours for mission de-tails; number of tikinas visited, sorties gener-ated, and packages delivered for mission re-sults; and total cost, fuel required, and person-nel required for mission resources. The usercan change CONOPS, dial-in technology combi-nations through vehicle parameters and cost pa-rameters of selected vehicles to see the rapid re-sponse of the measure of effectiveness from sur-rogate models as described above. Through theevaluation of alternatives in this parametric en-vironment, the user can determine the vehicleacquisition requirements that meet the intendedoperational effectiveness goals for future aircraftfleets.

5 Conclusion

A framework has been created that expandsupon traditional technology assessment tech-niques, like TIES, to simultaneously examine theimpact of system-level technology infusion andsystem of systems mission effectiveness for al-ternative tactics. A decision maker can benefitfrom the real-time sensitivity analysis and tradesperformed parametrically. A proof-of-concept

11


Fig. 12 Decision Support Environment.

demonstration was shown with a HADR mission,evaluating alternative vertical lift asset acquisi-tions for the Australian military. The system andSoS level surrogate models created from the DESwere integrated into the parametric dashboard, al-lowing the user to change the CONOPS, dial-intechnology combinations of different VTOL as-sets and immediately see the impact on MoEs.By evaluating the MoEs and MoPs of the hier-archical analysis environment, this approach canbe used to define requirements for system acqui-sition or new system development.

References

[1] J. Bergey, S. Blanchette Jr, P. Clements,M. Gagliardi, J. Klein, R. Wojcik, and B. Wood,“Us army workshop on exploring enterprise,system of systems, system, and software ar-chitectures,” tech. rep., CARNEGIE-MELLONUNIV PITTSBURGH PA SOFTWARE ENGI-NEERING INST, 2009.

[2] D. N. Mavris and M. R. Kirby, “Technol-ogy identification, evaluation, and selection forcommercial transport aircraft,” 1999.

[3] C. Taylor and O. D. Weck, “Coupled vehicle de-sign and network flow optimization for air trans-portation systems,” Journal of Aircraft, vol. 44,no. 5, pp. 1479–1486, 2007.

[4] D. P. Schrage, “Technology for rotorcraft af-fordability through integrated product/processdevelopment (ippd),” 1999.

[5] D. N. Mavris and D. A. DeLaurentis, “Method-ology for examining the simultaneous impact ofrequirements, vehicle characteristics, and tech-nologies on military aircraft design,” 2000.

[6] A. Borshchev and A. Filippov, “From systemdynamics and discrete event to practical agentbased modeling: reasons, techniques, tools,” inProceedings of the 22nd international confer-ence of the system dynamics society, vol. 22,Citeseer.

[7] J. Banks, Handbook of simulation: principles,methodology, advances, applications, and prac-

12


tice. John Wiley & Sons, 1998.[8] A. Skoogh, B. Johansson, and E. J. Williams,

“Constructive alignment in simulation educa-tion,” in Proceedings of the Winter SimulationConference, p. 137, Winter Simulation Confer-ence.

[9] R. R. Hill, J. O. Miller, and G. A. McIntyre,“Simulation analysis: applications of discreteevent simulation modeling to military prob-lems,” in Proceedings of the 33nd conferenceon Winter simulation, pp. 780–788, IEEE Com-puter Society, 2001.

[10] G. Weiss, Multiagent systems: a modern ap-proach to distributed artificial intelligence. MITpress, 1999.

[11] R. Ingalls, M. Rossetti, J. Smith, and B. Pe-ters, “Military applications of agent-based sim-ulations,”

[12] N. R. Jennings, K. Sycara, and M. Wooldridge,“A roadmap of agent research and develop-ment,” Autonomous agents and multi-agent sys-tems, vol. 1, no. 1, pp. 7–38, 1998.

[13] S. F. Railsback, S. L. Lytinen, and S. K. Jack-son, “Agent-based simulation platforms: Re-view and development recommendations,” Sim-ulation, vol. 82, no. 9, pp. 609–623, 2006.

[14] C. Weit, S. Chetcuti, L. Wei, M. Muehlberg,C. Chan, H. Gilani, C. Clanchy, K. N. Schwartz,A. Sudol, J. C. Tai, et al., “Modeling airlift op-erations for humanitarian aid and disaster re-lief to support acquisition decision-making,” in2018 Aviation Technology, Integration, and Op-erations Conference, p. 4007, 2018.

[15] R. H. Myers, Response surface methodology;Process and product optimization using de-signed experiments. No. 04; QA279, M84.,1995.

6 Contact Author Email Address

mailto: [email protected]

7 Acknowledgements

The authors would like to thank Colby J.Weit, Steven Chetcuti, Cherlyn Chan, MarcMuehlberg, Lansing Wei, Hassan Gilani, andKatherine G. Schwartz for their work on the

HADR mission example. Details of this casestudy can seen in Ref. [14].

Copyright Statement

The authors confirm that they, and/or their companyor organization, hold copyright on all of the origi-nal material included in this paper. The authors alsoconfirm that they have obtained permission, from thecopyright holder of any third party material includedin this paper, to publish it as part of their paper. Theauthors confirm that they give permission, or have ob-tained permission from the copyright holder of thispaper, for the publication and distribution of this pa-per as part of the ICAS proceedings or as individualoff-prints from the proceedings.

13

Documents

FORMULATION AND IMPLEMENTATION OF A METHOD FOR … · FORMULATION AND IMPLEMENTATION OF A METHOD FOR TECHNOLOGY EVALUATION OF SYSTEM OF SYSTEMS Fig. 2 Technology Identiﬁcation,