RT-171: Mission Engineering Competencies Technical Report · RT-171: Mission Engineering Competencies Technical Report Technical Report SERC-2018-TR-106 April 30, 2018 Principal Investigator:

RT-171:MissionEngineeringCompetencies

TechnicalReport

TechnicalReportSERC-2018-TR-106April30,2018

PrincipalInvestigator:Dr.GreggVesonder,StevensInstituteofTechnology

Co-PrincipalInvestigator:Dr.DineshVerma,StevensInstituteofTechnology

ResearchTeam:

StevensInstituteofTechnology:Dr.NicoleHutchison,Mr.SergioLuna,Mr.WilliamMiller,

Dr.HoongYanSeeTao,Dr.JonWade

Sponsor:OfficeoftheDeputyAssistantSecretaryofDefenseforSystemsEngineering(DASD(SE))

Report No. SERC-2018-TR-106 April 30, 2018

ii

Copyright©2018StevensInstituteofTechnology,SystemsEngineeringResearchCenterThe Systems Engineering Research Center (SERC) is a federally funded University Affiliated ResearchCentermanagedbyStevensInstituteofTechnology.Thismaterial is based uponwork supported, inwhole or in part, by theU.S.Department ofDefensethroughtheOfficeoftheAssistantSecretaryofDefenseforResearchandEngineering(ASD(R&E))underContractHQ0034-13-D-004.Anyviews,opinions,findingsandconclusionsorrecommendationsexpressedinthismaterialarethoseoftheauthor(s)anddonotnecessarilyreflecttheviewsoftheUnitedStatesDepartmentofDefensenorASD(R&E).NoWarranty.ThisStevensInstituteofTechnologyandSystemsEngineeringResearchCenterMaterialisfurnishedonan“as-is”basis. StevensInstituteofTechnologymakesnowarrantiesofanykind,eitherexpressedorimplied, as to any matter including, but not limited to, warranty of fitness for purpose ormerchantability, exclusivity, or results obtained from use of the material. Stevens Institute ofTechnologydoesnotmakeanywarrantyofanykindwithrespecttofreedomfrompatent,trademark,orcopyrightinfringement.Thismaterialhasbeenapprovedforpublicreleaseandunlimiteddistribution.


iii

TABLEOFCONTENTS

TableofContents..................................................................................................................iii

ListofFigures.........................................................................................................................v

ListofTables.........................................................................................................................vii

Acknolwedgements................................................................................................................8

ExecutiveSummary................................................................................................................9

1.BackgroundandOverview................................................................................................101.1PurposeoftheResearch...........................................................................................................131.2ResearchMethod.....................................................................................................................141.3ReportStructure......................................................................................................................16

2:MissionEngineeringContext.............................................................................................182.1WhatisaMission?...................................................................................................................18

2.1.1USDepartmentofDefense(DoD)MissionExamples..............................................................202.1.2USGovernmentAgencyMissionExamples(Non-DoD)...........................................................222.1.3NationalMissionInitiatives(USandNon-US)..........................................................................232.1.4 Critical Areas for Additional Research inMission Engineering Identified fromOpen SourceLiterature..........................................................................................................................................23

2.2DefiningMissionEngineering...................................................................................................242.2.1MissionAnalysis.......................................................................................................................262.2.2CapabilityEngineering.............................................................................................................262.2.3ServicesViewofSystemofSystemsEngineering....................................................................272.2.4InterviewResultsontheDefinitionofMissionEngineeringbyIndividualParticipants...........272.2.5 InterviewResultson theOrganizationDefinitionandPhilosophyonMissionEngineeringbyIndividualParticipants.......................................................................................................................312.2.6ResearchFindingsontheScopeofMissionEngineering.........................................................34

2.3AcademicPrograminMissionEngineering...............................................................................422.4ResearchFindingsontheGapsinMissionEngineering.............................................................43

3:MissionEngineeringCompetency.....................................................................................493.1MissionEngineeringCompetencyFramework..........................................................................49

3.1.1AREA1:DisciplineandDomainFoundations...........................................................................513.1.2AREA2:MissionConcept.........................................................................................................543.1.3AREA3:SystemsEngineeringSkills..........................................................................................553.1.4AREA4:SystemsMindset........................................................................................................563.1.5AREA5:InterpersonalSkills.....................................................................................................583.1.6AREA6:TechnicalLeadership..................................................................................................60

3.2TailoringtheCompetencyFramework......................................................................................623.3CompetencyAssessments........................................................................................................643.4MissionEngineeringTeams......................................................................................................673.5ComparisontoRelatedCompetencyModels............................................................................68

3.5.1MissionEngineeringCompetencyandtheHelix(Atlas)SEProficiencyModel........................683.5.2MissionEngineeringCompetencyComparedtoAdditionalCompetencyModels..................72

4.ViewsonFutureDirectionsforMissionEngineering..........................................................75


iv

4.1MissionEngineers’PerspectivesontheFuture.........................................................................754.2MissionEngineer’sFutureVisionforMissionEngineering........................................................804.3MissionEngineer’sPerspectivesonFutureChallenges.............................................................80

Conclusions..........................................................................................................................82

AcronymsandGlossary........................................................................................................83Acronyms......................................................................................................................................83Glossary.........................................................................................................................................84

AppendixA:RT-171PublicationsandPresentations.............................................................85

AppendixBCitedandRelatedReferences............................................................................86

AppendixC:DetailedMethodology......................................................................................91C.1 RT-171ResearchProcess......................................................................................................91

C.1.1 PreparationforDataCollection(A)......................................................................................92C.1.2 DataCollection(B)................................................................................................................92C.1.3 DataAnalysis(C)...................................................................................................................94C.1.4 AnswerResearchQuestions(D)...........................................................................................97C.1.5 PublishResults(E)................................................................................................................97C.1.6 MethodologyReview(F)......................................................................................................97

C.2Dataset....................................................................................................................................97

AppendixD:LiteratureReview.............................................................................................99D.1 SystemsEngineeringandSystemofSystems:DefinitionandScope......................................99D.2 SystemofSystemsEngineering:DefinitionandScope........................................................100D.3 CapabilitiesEngineering.....................................................................................................104

D.3.1 Perspectives.......................................................................................................................104D.3.2 ServicesViewofSystemofSystemsEngineering...............................................................105

D.4 RelationshipbetweenSystemofSystemsandMissionEngineering....................................105

AppendixE:InterviewResultsonCompetency...................................................................109E.1TechnicalCompetencies.........................................................................................................109

E.1.1DisciplineandDomainFoundations.......................................................................................110E.1.2MissionConcept.....................................................................................................................112E.1.3SystemsEngineeringSkills.....................................................................................................113

E.2SystemsMindset....................................................................................................................115E.3“Non-Technical”Competencies..............................................................................................116

E.3.1InterpersonalSkills.................................................................................................................116E.3.2TechnicalLeadership..............................................................................................................117

AppendixF:ExistingAcademicProgramsinMissionEngineering........................................119

AppendixG:MissionEngineeringProgramatOldDominionUniversity..............................121

AppendixH: ISO/IEC/IEEE152882015Systemsandsoftwareengineering–systemlifecycleprocesses............................................................................................................................123


v

LISTOFFIGURES

Figure1.Percentageofparticipantsbyorganizationtype...........................................................15

Figure2:MissionEngineeringwithintheSystemsEngineering‘V’Model(Moreland2015).......25

Figure 3: Percentage of Excerpts and Interviewees Based on their Definitions of MissionEngineering...................................................................................................................................28

Figure4:PercentageofExcerptsontheDefinitionofMissionEngineeringbyOrganizationType......................................................................................................................................................30

Figure5:InterviewResponse:“MEisSE+”CodingbyOrganizationType....................................31

Figure6:PerspectivesontheOrganizationDefinitionandPhilosophyonMissionEngineering.32

Figure7:MissionEngineeringCriticalActivities...........................................................................34

Figure8:ProcessesandPracticesinMissionEngineering............................................................38

Figure9:InterviewResponsesontheOverlapofMissionEngineeringandSystemsEngineering......................................................................................................................................................40

Figure10:InterviewResponsesfromthosewhoindicatethatthereisOverlapbetweenMissionEngineeringandSystemsEngineeringWork................................................................................41

Figure 11: Summary of the Interview Responses on the Critical Challenges in MissionEngineering(N=32).......................................................................................................................43

Figure12:CriticalNon-TechnicalImplementationChallengesinMissionEngineering...............44

Figure13:CriticalTechnicalChallengesinMissionEngineering..................................................46

Figure14.MissionEngineeringCompetencyFramework............................................................50

Figure15.ExampleCompetencyProfileforanIndividual............................................................66

Figure16.ExampleCompetencyProfilewithTargetLevels.........................................................67

Figure17.Exampleofteamprofilescomparedtoan“expected”profile...................................68

Figure18.ComparisonofMECompetencyFrameworkwithOtherCompetencyModels..........73

Figure19.Distributionofresponsesforfuturedirectioninmissionengineering........................75

Figure20.Distributionofresponsesforvisionofmissionengineering.......................................76

Figure21.Distributionforresponsestowhathastochangetomakeitareality.......................77

Figure22.Distributionofresponsestorisksinmissionengineering...........................................78

Figure23.Distributionofresponsestoobstaclesinobtainingthecompetencies.......................79

Figure24.RT-171ResearchProcess.............................................................................................92

Figure25.ExampleofCodingRelationships.................................................................................96

Figure26.ExampleofLevelsofCoding........................................................................................96


vi

Figure27.PercentageofIntervieweesbyOrganizationType......................................................98

Figure28:ArtifactsintheContextofCoreElementsofSoSSE(Dahmann2010)......................102

Figure29:SoSWaveModel(Dombkins2008andDahmannetal.2011)..................................103

Figure30.RelationshipbetweenMissionEngineeringandSystemofSystemsEngineering.....108

Figure 31. Percentage of Competency Excerpts around Specific Types of Competencies(N=1834).....................................................................................................................................109

Figure32.TechnicalCompetenciesExerptsAggregatedintoCompetencyAreas(N=847).......110

Figure33.DisciplineandDomainCompetencies(N=32interviewees;214excerpts)...............111

Figure34.MissionConceptCompetencies(N=32interviewees;235excerpts)........................113

Figure35.SystemsEngineeringCompetencies(N=32interviewees;380excerpts)..................114

Figure36.SystemsMindsetCompetencies(N=29interviewees;157excerpts)........................115

Figure37.InterpersonalSkillsCompetencies(N=30interviewees;199excerpts).....................117

Figure38.TechnicalLeadershipCompetencyCategories(N=30individuals;139excerpts)......118


vii

LISTOFTABLES

Table1:SampleMissionAreasforMissionEngineeringWorkwithintheUSDoD......................13

Table2:MissionEngineeringExamplesbyotherGovernmentAgencies.....................................19

Table3:SelectedInterviewExcerptsontheOrganization'sDefinitionandPhilosophyonMissionEngineering...................................................................................................................................33

Table4:SelectedInterviewExcerptsonCriticalActivitiesinMissionEngineering......................35

Table5:InterviewExcerptsonProcessesandPracticesinME....................................................38

Table6:SelectedInterviewExcerptsontheOverlapbetweenSystemsEngineeringandMissionEngineering...................................................................................................................................41

Table7:SelectedInterviewExcerptsonCriticalNon-TechnicalImplementationChallengesinME......................................................................................................................................................44

Table8:SelectedInterviewExcerptsonCriticalTechnicalImplementationChallengesinME...46

Table9.TailoringtheMissionEngineeringCompetencyModel..................................................62

Table10.CompetencyLevels(adaptedfromPysteretal.2018,inprint,usedwithpermission)......................................................................................................................................................65

Table 11. Comparison of Mission Engineering Competency Framework and Helix (Atlas)ProficiencyModel.........................................................................................................................69

Table 12: Comparison between Systems and Systems of Systems as Applied to SystemsEngineering(DahmannandBaldwin2008andNeagaet.al.,2009)..........................................101

Table13:SoSEWaveModelAppliedtoMissionEngineering(Dahmannet.al.2018)..............103

Table14.AcademicProgramswithMissionEngineering-RelatedCourses................................119

Table15:CoursesandDescriptionsofODU’sGraduateCertificate inMissionEngineeringandAnalysis.......................................................................................................................................121


8

ACKNOLWEDGEMENTS

The Mission Engineering Competency team would like to thank the 32 individuals whoparticipated in the project, offering their resources, time, and effort. Thiswas critical to ourresearch.Theiractiveparticipationininterviewsprovidedusdatathatwasrichinbothqualityandquantity,whichmakesthisresearchmorevaluableandusefultothemissionengineeringcommunityatlarge.Wearemostgrateful to theOfficeof theDeputyAssistantSecretaryofDefense forSystemsEngineering(DASD(SE)),especiallyKristenBaldwin,RobGold,ScottLucero,andAileenSedmakfortheirsupport,withoutwhichthisresearchwouldnotbepossible.TheMECompetencyteamwouldespeciallyliketothankDavidHeckmanandJudithDahmannfortheirsupport,outreach,andguidancethroughoutthisstudy.


9

EXECUTIVESUMMARY

Missionengineeringisthedeliberateplanning,analyzing,organizing,andintegratingofcurrentand emerging operational and system capabilities to achieve desired warfighting missioneffects. (DAG2017).Thoughsystemsacquiredwithin theDepartmentofDefense (DoD)havehadamissioncontext, theyhaveoftenbeenacquired individuallyandnotaspartofa largermission.Thefocusonacquiringsystemsandsystemsofsystemsinawaythatsupportslargermissionsisanemergingarea.

Supportedbya literaturereviewofmissionengineeringandrelatedareassuchassystemsofsystemsandcapabilityengineering,theresearchteamhasinterviewed32individualswhoareor have beenmission engineers. The views of the mission engineering workforce provide acrucial perspective on the emerging area of mission engineering, in particular, the skillsetswhich characterize mission engineering competencies. The DoD has defined ‘missionengineering’, but there is a range of differing views of the definition and scope of missionengineering and its relationship to systems engineering among current practitioners. Thedifferences inviewsare reflected in this report, includingperspectives fromUSorganizationsoutsidetheDoDaswellasnon-USorganizations.Itshouldbenotedthatmissionengineeringisanemergingdisciplineandthisreportreflectsthecurrentstateofitsmaturity.

The current core competencies identified by today’s mission engineers overlap withcompetencieswhicharepartofthesystemsengineeringcompetencybasefromtheAtlas/Helixresearch, butwith added emphasis on key areas, particularly in terms of the criticalmissioncontext and operational environment and systems of systems perspectives. (Hutchison et al.2018) The key competency areas are: Discipline & Domain Foundations, Mission Concept,SystemsEngineeringSkills,SystemsMindset,InterpersonalSkills,andTechnicalLeadership.


10

1.BACKGROUNDANDOVERVIEW

Thisreportprovidestheresultsofa16-monthstudyonmissionengineeringconductedbytheSystemsEngineeringResearchCenter(SERC).TheSERCwastaskedbytheOfficeoftheDeputyAssistantSecretaryofDefenseofSystemsEngineering (DASD(SE))withexaminingthecurrentstateofmissionengineeringpracticewithintheDoD.

MissionengineeringintheUSDoDisarelativelynewendeavor.ThekeyUSDoDpolicydrivingtheresearchonmissionengineeringistheMissionIntegrationManagement(MIM)legislationin the National Defense Authorization Act (NDAA) for Fiscal Year 2017 Section 855 (NDAA,2017).Therecommendedmissionareasinclude:

1. Closeairsupport

2. Airdefenseandoffensiveanddefensivecounter-air

3. Interdiction

4. Intelligence,surveillance,andreconnaissanceand

5. Anyotheroverlappingmissionareaofsignificance,asjointlydesignatedbytheDeputySecretaryofDefenseandtheViceChairmanoftheJointChiefsofStaffforpurposesofthissubsection.

TheresponsibilitiesoftheMIMactivitiesforamissionareaincludethefollowing:

1. Developingthetechnicalinfrastructureforengineering,analysis,andtest,includingdata,modeling,analytictools,andsimulations;

2. Conducting tests, demonstrations, exercises, and focused experiments for compellingchallengesandopportunities;

3. Overseeing the implementationof Section2446cof title 10 code,United StatesCode(requirementsdiscussedbelow);

4. Sponsoring and overseeing research on and development of (including tests anddemonstrations)automatedtoolsforcomposingsystemsofsystemsondemand;

5. Developingmission-basedinputsfortherequirementsprocess,assessmentofconcepts,prototypes, design options, budgeting and resource allocation, and program andportfoliomanagement;and

6. Coordinatingwithcommandersofthecombatantcommandsonthedevelopmentofconceptsofoperationandoperationalplans.

Section2446coftitle10code,UnitedStatesCode(10USC2446c)referstotherequirementsrelating to availability of major system interfaces and support for modular open systemapproachandprototyping.TheAcquisitionAgilityAct intheNDAAFY17Sections805-809putthe10USC2446c inplace.TheMIMresponsibilities inSection855regardingmanagementofinterfaces include overseeing the implementation of Section 805. The MIM activities for a


11

mission area shall extend to the supporting elements for the mission area, such ascommunications,commandandcontrol,electronicwarfare,andintelligence.InregardstotheUSJointStaff,thekeyUSDoDpolicyformissionengineeringistheJointCapabilityIntegrationandDevelopmentSystems(JCIDS) instructionmanual in2015,whichmandatesmission-basedassessmentsandsystemsinteroperabilityacrossUSDoDandcomponents.

In2016,theActingDeputyAssistantSecretaryofDefenseforSystemsEngineering(DASD(SE)),Ms.KristenBaldwinstartedaseriesofdiscussionsonthetopicofmissionengineeringattheUSDoDSystemsEngineering (SE)Forum.The intentof thesemeetingswastobeginadiscussionbetween the offices in theOffice of the Secretary ofDefense (OSD), the Joint Staff, and theorganizations performingmission engineering (Gold, 2016). Thepurposeof these roundtablemeetingswastoidentifypolicy,organizations,methods,tools,challengesandopportunitiesformission engineering enterprise improvements. The series of discussions concluded in anenterprise-level discussion to synthesize mutual approaches, challenges, and potentialrecommendationsfortheacquisitioncommittee.

The roundtables were conducted by the DASD(SE) with the following organizations: Army,Navy, Air Force, Missile Defense Agency (MDA), and the Joint Staff. The outcomes were asfollows:

• Understanding of current practices: processes, techniques, tools, measures, and theroleofmodeling&simulationandtest&evaluation.

• Identification of actions in policy, resources, and research to affect improvements tocurrent practices; common, persistent challenges, gaps, or obstacles requiringattention;andtheinitialsenseofhowDoDseesworkingwithindustry.

• DevelopmentofabriefingpackageforDoDcomponentleadershipandUSD(AT&L),thelattereffectivein2018tobeUSD(R&E)andUSD(A&S).

The Army mission engineering focus, briefed at the roundtable in 2016, was on integratingupdatednetworksystemsintocapabilitysetsfordeployment.Beginningin2017,theArmyhasstoodupinitiativesundertheaegisoftheFuturesCommandwithcross-functionalteams(CFTs)addressingthefollowingsystemsofsystems:1)long-rangeprecisionfires;2)nextgenerationcombat vehicles; 3) future vertical lift; 4) air and missile defense; 5) soldier lethality; 6)synthetic training environment; 7) network, command, control, communications andintelligence;and8)precision,navigation,andtiming.

TheNavyfocusin2016was,andremains,integrationandinteroperability(I&I).ThisisaNavy-wideinitiativeforanalysisofnavalmissionstounderstandhowwellcurrentsystemsmeetNavymissionneeds,andtoidentifygaps.TheNavyseekstounderstandintegration/interoperabilityissues between systems and identify where investments are needed to improve missionperformance. I&I considers both material and non-material (operational) solutions usingDoctrine, Organization, Training, Materiel, Leadership and Education, Personnel, Facilities(DOTMLPF). The Navy is in conformance with the Joint Capabilities Integration and


12

Development System (JCIDS) and Department of DefenseInstruction (DoDI) 5000.02 for systems requirements andacquisition.TheI&Iapproachistousenavalmissionthreadsfromasystemofsystems(SoS)mission-levelperspective,forthe purpose of identifying and prioritizing changes andupgradesinthesystemssupportingthemissions.

TheAir Force focus is to identifymission sets alignedwiththe Joint Simulation Environment for the F-35 aircraft. TheAir Force is developing a fifth generation modeling andsimulation enterprise. There are five coremissions and 42 sub-mission sets; the intent is toperformmissionthreadanalysisandassessmentacrossthemissionsetstoassesskeysystemsand risks. The Air Force also adheres to JCIDS and DoDI 5000.02 for requirements andacquisition,usingvariousanalyticaltechniquesintheirengineeringanalysis.

The MDA has a systems engineering office with end-to-end responsibility for the BallisticMissileDefenseSystem(BMDS)withelementsthataresystemsintheirownrightmanagedbybothMDAandtheservices.MDAisexemptedfromJCIDSandDoDI5000.02,allowingthemtofocus their engineeringon the set of systems supporting the end-to-endmission; this allowsMDA to make trades across systems to meet mission needs from the initial definitionassociated with each new increment. Hence there is a strong emphasis and substantialinvestmentinmodelingandsimulation.

TheJointStaffhasdevelopedgenericmissionthreadstoperformmissionanalysis.Thefocusison joint mission integration using joint mission threads as the foundation for 1) improvedinteroperabilityinkeyareassuchascloseairsupportandjointfires,and2)integrationofforcesfor operations including interoperability assurance and validation of coalition forces. Themissionanalysisincorporatesadversarycapabilityinteractionsperformedmanuallyinhouseorleveragingotherorganizations’modelingandsimulationcapabilities.

Inassociationwiththegovernmentroundtables,industryformedataskforcetoassesscurrentindustry activities and viewpoints on mission analysis and mission engineering. Supportingorganizationsforthissurveyandassessment,ledbytheNationalDefenseIndustrialAssociation(NDIA) Systems Engineering Division (SED) and the International Council on SystemsEngineering (INCOSE), included Military Operational Research Society (MORS), Institute ofElectrical and Electronics Engineers (IEEE), Aerospace Industries Association (AIA), andAmerican Institute of Aeronautics and Astronautics (AIAA). The conclusions of the industrysurveyandassessmentisthatindustryfindsvalueinmissionengineeringandmissionanalysis.Industry has a large number of practitioners who use a variety of approaches and tools.Industry respondentsdesire toworkmorecollaborativelywithDoDto refineandunderstandthe definition ofmission engineering and address common challenges including sharing bestpractices,tools,andmodels;findameanstoprovideaccesstorelevantdata;shareresourcesfor skill development; and recommend the establishment of a joint action plan to moveforward.

Throughoutthisreport,systemsofsystems are mentioned. A singlesystem of systems is abbreviatedSoS, usually when referencing aspecific system of systems. Ifmultiple systems of systems or ifthe general concept of systemsofsystemsarereferenced,thiswillbenotedasSoS’s.


13

Based on the literature review, several examples ofmission engineeringworkwithin theUSDoD, other government agencies, the US, and capabilities engineering by other nations arehighlightedinTable1,below.

Table1:SampleMissionAreasforMissionEngineeringWorkwithintheUSDoDCurrentStatus Mission(s)

CurrentlyAddressed

• BallisticMissileDefense(MDA)• NuclearCommandandControl/NationalLeaderCommandand

Control(NLCC)• DigitallyAidedCloseAirSupport(DACAS)• Air/CruiseMissileDefense(NavyAEGISandArmy)• IntegratedAirMissileDefense(IAMD)

CrossServicesExamples

• TacticalSATCOM• CHEMBIO• EnvironmentalMonitoring(Weather)• SpectrumOperations• AssuredPosition,Navigation,andTiming(PNT)• CyberSituationalAwareness

NeededMissionEngineeringApproaches

• AirSuperiorityinContestedEnvironments• WideAreaSurveillanceandTargeting

InTable1,“NeededMissionEngineeringApproaches”areengineeringapproachesthatarenotofficially designated as mission engineering, but which many share characterstics with themissions identified for this research (addresses a complex problem requiring a system ofsystemssolutionthatcrossesorganizationalboundaries).

1.1PURPOSEOFTHERESEARCH

Thepurposeof this initiative is todevelopamodelof thekeycompetencies required for theDoDacquisitionworkforcetosupportmissionengineering.ThecompetencymodelwillincludeskillsandexperiencesnecessarytoperformthefollowingmissionengineeringactivitiesacrosscomplexsystemsandSoS’sincluding,butnotlimitedto:missionanalysisandsynthesis,trade-off analyses, technologymanagement, resourcemanagement, architecture development andmodeling, mission modeling, addressing supporting capabilities (e.g., communications) andoverarchingmissionfunctions,synchronizationoftestingandindividualsystemimplementation(based on the Department’s established ‘Systems of SystemsWaveModel’ and the current‘SystemsEngineeringGuideforSystemsofSystems’)aswellasreflectindustryapproachesandbestpractices.This initiativehasactive interestamongandcommitmentof support fromtheArmy,Navy,AirForce,MDA,andDefenseAdvancedResearchProjectsAgency(DARPA),eachofwhomhavebeenperformingvariousformsoftheaboveactivities insupportoftheiragency-specific mission needs. The US DoD defines mission engineering as the development anddeployment of a military capability by applying a mission context to SoS and to complexsystems within the Department. Based on the findings of the research, the RT-171 teamrecommendsabroaderview:


14

Missionengineeringdiffersfromsystemsengineeringbecauseitnecessarilyincludesasystemof systems context: the individual systems that comprise themilitary capability (e.g. shipsoraircraft)areinherentlyflexible,functionallyoverlapping,multi-missionplatformssupportedbya complex backbone of information communication networks. The composition of assetsperforming missions are dynamic, both spatially and temporally (Garrett et al 2011 andMoreland and Thompson2017). This context is unlike traditional systemsengineeringwherethere is little tono functionoverlapor flexibilitybecause individual functionsaremapped toonly one element in the system. SoShas arisen in response to increasingly complex systemsbeingdevelopedbytheDepartmentwherethecapabilitiesofthemultiple linkedsystemsaregreaterthanthesumofthecapabilitiesoftheconstituentparts.Themissioncontext isakeyelementtoassistingdevelopersandmanagerstodeterminewhichsystemshavetobeinvolved,whatfunctionstheyhavetoperform,andhowoperators/userswillmakeuseofthesesystems.With the emergence of a mission-focus to SoS efforts over the last five to six years, theengineering community is now able to successfully assess and determinewhich systems arerelevanttoacapabilityandhowtomodifythosesystemstosupportcriticalmissionareassuchas air defense and offensive and defensive counter-air; and intelligence, surveillance, andreconnaissance.

1.2RESEARCHMETHOD

The research is based on a mixed-methods approach, utilizing grounded theory to extractmeaning fromdatacollected in interviewsaswellasa traditional literaturereview.Basedonrecommendations from members of the Officer of the Secretary of Defense and identifiedadditional interviewees through literature review and recommendation from studyparticipants.Theteaminterviewed32individualswhoarecurrentlyorwererecentlypracticingmissionengineers.These individuals camepredominantly from theUSDoD, thoughnon-DoDUS government, US commercial, and non-US government entities were also included. (SeeFigure1)TheinterviewquestionscanbefoundinAppendixC.Thetraditionalliteraturereviewfocused on people, processes, methods, and tools used to perform, or propose, missionengineering.Theliteraturereviewfindingsareusedtocorroboratetheresponseselicitedfromtheinterviews.Theteamperformedqualitativeanalysisontheinterviewdata,primarilycodingforlikegroupsandthendevelopingadditionalstructurebasedonthecontentofdata,allowingthethemestoemerge rather than starting with an expected framework (grounded theory). The results ofthesequalitativeanalysesarepresentedthroughoutthisreport.Datagroupingsincluded:

MissionEngineeringcombinesthestructureofsystemsengineeringandthetacticalinsightsofoperationalplanningtoasystemofsystemstodeliveraspecificcapability.


15

• Definitionofmissionengineering

• Relationshipbetweenmissionengineeringandsystemsengineering

• Currentpracticesinmissionengineering

• Currentchallengesinmissionengineering

• Criticalskillsformissionengineering

• Expectedfuturechallengesformissionengineering

Figure1.Percentageofparticipantsbyorganizationtype.

Anothercommonthemeinthedatawastheimportanceofsystemsofsystemsengineeringformission engineering; however, this is detailed at the next level of analyses below the abovegroupings.Inaddition,theteamanalyzedover50sourcesfortheirliteraturereview,coveringtopicssuchasmissionengineering,systemofsystemsengineering,capabilityengineering,andforce design. The results of the literature review are highlighted in Appendix D as well asintegratedintothediscussionsofresearchfindingsasappropriate.

31%

28%

16%

13%

9%

3%

0%

5%

10%

15%

20%

25%

30%

35%

Navy Army DoD(General) Non-Government

Non-DoDGovernment

AirForce

PercentageofParBcipantsbyOrganizaBonType(N=32)

OrganizaBonType


16

1.3REPORTSTRUCTURE

Thebodyofthisreportisdividedintothreeadditionalsections:• Section2:MissionEngineeringContext–Basedprimarilyonthe interviewresponses,

this sectionprovidesanoverviewofhowmissionengineering is conductedwithin theDoDandalsoreflectslessonslearnedfromotherUSgovernmentagenciesandindustrialorganizations. In addition, this section provides an overview of existing academicprogramsfocusedonmissionengineering.

• Section 3: Mission Engineering Competency Framework – based on all the datacollected to date, including the literature review (AppendixD) and the data collected(Appendix E). This section presents a competency framework tailored specifically tomissionengineering.

• Section 4: Future Directions for Mission Engineering – based on the interview datacollected,thissectionprovidesperspectivesonhowmissionengineeringisexpectedtoevolve.

Thebodyof this report is intended tobe conciseand streamlined.However, theappendicesprovide supporting information about how the conclusions in the body of the report werecreated:

• AppendixA:PublicationList–ThisappendixprovidesalistofpublicationsbytheRT-171teamrelatedtothisresearchforadditionalinformation.

• AppendixB:References–Thisappendixprovidesalistofallmaterialsreferencedandreviewedaspartoftheresearchproject.

• AppendixC:Methodology–ThisappendixprovidesthedetailedmethodologyusedbytheRT-171teamtoconducttheresearch.

• AppendixD:LiteratureReview–Thisappendixprovidesanoverviewofthecriticalliteraturereviewedaspartoftheresearchproject.

• AppendixE:InterviewDataAnalysis–ThisappendixprovidesresultsofthedetailedqualitativeanalysisconductedbytheRT-171teamontheinterviewdatacollectedthroughouttheproject.

• AppendixF:ExistingAcademicProgramsinMissionEngineering–ThisappendixprovidesalistingofallprogramsthathavemissionengineeringrelatedcurriculaidentifiedbytheRT-171team.

• AppendixG:MissionEngineeringProgramatOldDominionUniversity–ThisappendixprovidesanoverviewoftheonlymissionengineeringdegreeprogramidentifiedbytheRT-171team.


17

• AppendixH:(ISO/IEC/IEEE)15288guidelines–Thisappendixprovidesguidanceonthe‘Systemsandsoftwareengineering–systemlifecycleprocesses’standardpublishedjointlybytheInternationalStandardsOrganization(ISO),theInternationalElectrotechnicalCommission(IEC),andtheInstituteofElectricalandElectronicsEngineers(IEEE).


18

2:MISSIONENGINEERINGCONTEXT

Thissectionprovidesanoverviewofthestateofmissionengineeringfromtheperspectiveofthestudyparticipantatthetimeofpublication,withadditionalinformationfromtheliteraturereview.

2.1WHATISAMISSION?

There are many definitions for the word ‘mission’, from the colloquial – e.g. ‘an importantassignment’–tothosespecifictoagivencontextsuchasdefense,homelandsecurity,orspaceexploration.Thedefinitionof‘mission’forthepurposesofthisreportis:

TheUSNavyapproachtomissionengineeringchargesthenavalsystemscommands(SYSCOMs)“to place an increased emphasis on assessing the I&I (integration and interoperability)” ofwarfare systems to support current and future readiness for critical mission threads. Theassessment of naval technologies, systems and/or capabilities requires a system-of-systems(SoS) approach to analyze the impact ofmaking these naval investments across the diversedomains of surface, undersea, air, land, and networks as well as maritime coalition forceintegration.Theseassessmentsareexecutedfollowingasystematic,quantifiable,anditerativeapproach referred to as Mission Engineering, which combines the structure of systemsengineering and the tactical insights of operational planning. The findings are captured in‘effects/kill chains’ to clearly identify operational needs based on the way we plan to fightthroughmission threads captured in our Combatant Command’sOperational Plans (OPLANs)and Contingency Plans (CONPLANs). Mission Engineering emphasizes capability-basedassessmentstoproduceintegratedwarfightingcapabilitiesthatcanbetranslatedintospecificprogrammaticguidanceforstrategicprograms.”(Moreland2015).The MDA approach “revolves around the concept of a mission context which managesuncertainties, dynamics and stochastic behaviors of SoS’s. It has been posited that complexSoS’saredrivennotbytheperformanceandbehaviorsoftheconstituentcomponents,ratherthey are driven by the complex integration and interoperability, the interstitials, of thecomponentstoachieveMission-levelgoals…missionthreadsarethedescriptionoftheend-to-end set of activities and component systemsemployed to accomplish specific subsets of themissiongoalsandobjectives.”(DeiotteandGarrett2013).Ageneric‘killchain’missionthreadisexpressedasanevent-basedsequenceofoperationsonatimeline:detect,track,engage,assess,and(potentially)re-engage(Garrett,Anderson,Baron,andMoreland2011).Thegeolocationofthedifferentgenericoperationscanbeonthesameor

The task, togetherwith the purpose, that clearly indicates the action to be taken and the reasontherefore.(DoD2016)


19

distributedplatformsinterworkedtogether;thesegenericoperations,arecalled‘functions’or‘activities’ in systems engineering speak. The allocations of these functions or activities tophysical assets candynamically changeover the courseof amission thread. Theengineeringdesignofsystems,orsystemsofsystems,integratesthesetof(dynamicallychanging)missionthreads intoawholeusingexecutablemodelingmethods rooted ingraph theory thatcanbeassessedintermsofstructuralintegrity,behavior,performanceatscale,andresilience(BuedeandMiller2016).Whenmodeledandintegratedinthismanner,anindividual‘effects/killchain’missionthreadisjustoneinstantiationthroughamorecomplicatedorcomplexnetworkedwebofcapabilities.Table2providesanoverviewof several examplesofmissions reviewedby theRT-171 team.EachofthesemissionsiselaboratedinSection2.1.1–2.1.4,below.

Table2:MissionEngineeringExamplesbyotherGovernmentAgencies

OrganizationName Type Mission(s)

USDoD

EnterpriseIntelligence,Surveillance,Reconnaissance(ISR)

Joint,Multi-DomainUS Pacific Command (PACOM) JointOperations

MissileDefenseBallistic Missile Defense and TheaterMissileDefense

Service-SpecificNavy BallisticMissile Defense and Anti-AirWarfare

Service-SpecificNavy All Domain Offensive SurfaceWarfareCapabilityandNavalIntegratedFireControl-CounterAir(NIFC-CA)

Service-SpecificArmyCounterRussianHybridWarfare

Service-SpecificArmy Brigade Combat Team (BCT) AirandMissileDefense

FederalAviationAgency(FAA) National

NationalAirspaceSystem(NAS)

NASA InterplanetaryTravel JourneytoMars

USGovernment(cross-departmentinitiative)

InfrastructureCritical Infrastructure Protection andRecovery

AustralianMinistryofDefense(MOD) Defense Australian Land-Force Capability

IntegrationUKMinistryofDefence Defense GenericVehicleArchitecture(GVA)


20

2.1.1USDEPARTMENTOFDEFENSE(DOD)MISSIONEXAMPLES

ThefollowingareseveralexamplesofDoDMissionsthathavebeenanalyzedinthisresearch.IntelligenceSurveillanceReconnaissance(ISR)ISR providers – service-specific assets that provide intelligence, surveillance, andreconnaissance as well as national means provided by the intelligence community – viewthemselves as enterprises. The legacy structureof theseenterprises is basedona functionaldesign where individual systems are ‘owned' by the functional organizations. Capabilitydirectoratesareoverlaidonthefunctionalorganization;thecapabilitiesownersthinkintermsof mission threads to produce ISR products. This results in a natural tension between theowners of themission threads and the owners of the program systems. A services orientedarchitecture(SOA)isthecommonapproachtointegratethefunctionalsystemstoexecutethemission threads. The individual systems are viewed as Lego™ blocks to provide a reusablearchitectureforexecutingdifferentmissionthreads.A draft definition of the problem reflective of the complexity of ISR is as follows. In today’scontext of a complex, richly interconnected world, enterprises traditionally organized alongfunctional lines face cultural, organizational and technological challenges transforming tocapabilities-basedenterprisesoperatingat Internetspeeds.Their functionalmodelsevolve toidentifiableorganizationswithintheenterprise,eachseekingtooptimizetheirpartofasystemsof systems. The end result of the enterprise as viewed from the outside is dysfunctional atworstandinefficientatbest.Capabilities-basedenterprisesfacetheirownchallengesintermsof creating and sustaining rapid but inefficient custom mission threads versus efficientplatform-based mission threads with reusable components. Transformation from the legacyfunctional enterprise to the capabilities enterprise is hindered by cultural and institutionalinertia reinforced with aging installed technology bases. Culture, institutional inertia,incentives, and enabling technologies must all be transformed. The successes of thesetransformationsarelimited,buttherearesuccessmodels.USPacificCommand(PACOM)JointOperationsPACOMand its service components are proactivelyworking counter anti-access/aerial denial(A2/AD) and joint firesusinga fictional jointoperations vignettewhereahostilepowerwithsubstantialmilitary capabilities seizes controlofan island in the Indo-AsiaPacific regionwithwhich the US has treaty obligations requiring military intervention. The island is decisiveterrain1influencingaerialandmaritimenavigationoraccesstoastrategicport.Themissionistosecuretheislandandrestoreunhinderednavigation.Themissionsequencingisasfollows:1)cyberandspacecapabilitiestotemporarilyblindanddisruptthehostilepower’scommandandcontrolsystems;2)specialoperationsforcesinfiltratetheislandandtemplate2theopponent;

1Ageographicplace…that,whenactedupon,allowscommanderstogainamarkedadvantageoveranenemyorcontributemateriallytoachievingsuccess.(DoD2018)2 “Template” is used by the DoD in the context of characterizing/patterning the capabilities, doctrine, etc. ofopponents.


21

3)Marineamphibious forcessecurethe island includingbeachhead,airfield,andothermajorstructures;4)Armyengineersfollow-ontorepairtheairfieldandconstructhardeneddefensivepositions; 5) a reinforced Army Stryker battalion is air landed to replace theMarines and isaugmented with 155mm howitzers, HIMARS (rockets) equipped with anti-ship cruisemissilepods, and air andmissile defense assets to defend the island for an extendedperiod. TheseforcesareprotectedbyAirForcemannedandunmannedsystems,Navyships,andunderwaterdrones.BallisticMissileDefenseandTheaterMissileDefenseMDA has end-to-end responsibility for both the ballisticmissile defense system (BMDS) andtheatermissiledefense.TheBMDSistodefendtheUShomelandandUSregionalfriendsandalliesagainstlimitedballisticmissileattacks.TheBMDSisdesignedtocombinethecapabilitiesof the ground-basedmidcourse defense (GMD) systemwith a network of ground-, sea-, andspace-basedsensorstoprovideanintegrated,layereddefense.GMDdefendsagainstthreatsbylaunchingground-basedinterceptorsthatreleasekillvehiclestofindanddestroythethreat.Themissionengineeringapproachforbothballisticmissiledefenseandmissiledefensedefinesmission threads as kill chains instantiated as dynamic spatial and temporal graphs combinedwith agent-based models to analyze performance at scale. A critical element in missionengineeringinthisdomainismanagingtheinterstitials,thatis,theinterfaces,interoperability,andintegrationbetweenconstituentsystemsinthesystemofsystems.NavyBallisticMissileDefenseandAnti-AirWarfareNavyAegiscombatsystemsoncruiseranddestroyerplatformsprovideBMDS,theatermissiledefense, and anti-air capabilities, the latter against both manned and unmanned threatsystems. The mission engineering approach for missile defense and anti-air warfare definesmission threads as kill chains instantiated as dynamic spatial and temporal graphs combinedwith agent-based models to analyze performance at scale. A critical element in missionengineeringinthisdomainismanagingtheinterstitials,thatis,theinterfaces,interoperability,andintegrationbetweenconstituentsystemsinthesystemofsystems.ThereisclosesynergyintheMDAandNavyapproachestomissionengineering.Navy All Domain Offensive Surface Warfare Capability and Naval Integrated Fire Control-CounterAir(NIFC-CA)The offensive anti-surface capability ties targeting information from satellites, aircraft, ships,submarines,andtheweaponsthemselvesina‘tacticalcloud’toformakillweb.TheconceptissimilartothecarrierstrikegroupNIFC-CAinwhichaircraftandshipsinthestrikegroupsharetheirtargetinginformationonaircraftandcruisemissilethreatsviahigh-capacitydatalinkstoothershipsandaircraftthatmightbeoutofsensorrange,butnotoutofweaponsrangeofatarget.Again, the concepts leverage the kill chain approach from the Aegis program.ArmyCounterRussianHybridWarfareRussia has demonstrated sophisticated hybridwarfare approaches in theUkraine that blendand integrate disinformation campaigns, cyber warfare, insurgency, surveillance drones,


22

massiveartillery/rocketstrikesonopposingforceassemblyareas,andprotectedbyadenseairdefense system providing anti-access/aerial denial (A2/AD). The Army is performingmissionanalysisandengineeringtoclosethegapincapabilities.ArmyBrigadeCombatTeam(BCT)AirandMissileDefenseTheArmyBCTAirandMissileDefense initiative is toclosethegap inBCTairdefenseagainstmannedaircraft,UAS,cruisemissiles,rockets,artillery,andmortars.TheArmyisemployingthejointcounter-airframeworkusingattackoperationstofind,fix,anddefeatUASgroundstationsin conjunction with indirect fires. Passive air defense measures include cover, camouflage,concealment,anddeceptiontoreduceaerialobservation.Keyassetsarehardened,formationsdispersed,andredundancyisestablishedforkeynodestoreducetheeffectsofattack.Passivemeasures also include providing early warning to units. Active defense measures includeallocationofassetsfornon-dedicatedairdefense,anddedicatedairdefenseartillery.

2.1.2USGOVERNMENTAGENCYMISSIONEXAMPLES(NON-DOD)

The following are several examples of non-DoD US Government Missions that have beenanalyzedinthisresearch.FAANationalAirspaceSystem(NAS)The NAS is a complex system of systems integrating the control of manned and unmannedaircraftandrocketlaunchesandlandingsintheUSairspace.ThenextgenerationNASistoscaleto the increased number of objects in the airspace. The process for engineering the NAS isdocumented in the NAS Systems Engineering Manual driven by mission analysis to identifycapabilityshortfallsandthenengineersolutionstoaddresstheshortfall.NASAJourneytoMarsNASAhasamaturemissionplanninganddevelopmentcapabilityforengineeringthejourneytoMars, with experience in integration of spaceflight elements that spans across systems,vehicles, and programs, aswell as government, industry, and international partners.Missiondevelopmentcapabilitiesinclude:

• Productdevelopmentandverification,includingmissiontimelines,flightrules,andcrewandgroundprocedures

• Analysis and modeling of vehicle performance, consumables and human / vehicleinterfaces

• Flightdesignoflaunch,orbit,andentryflighttrajectoriesandassociatedriskcontrols• Operations tools design, development and testing, including onboard and ground

software• Missionreadinessassessments• SafetyandRiskAssessment.


23

2.1.3NATIONALMISSIONINITIATIVES(USANDNON-US)

The following are several examples of US and Non-US NationalMission initiatives that havebeenanalyzedinthisresearch.CriticalInfrastructureProtectionandRecoveryThe theme of the December 2016 issue of INSIGHT magazine for systems engineeringpractitioners published by INCOSE and Wiley was “critical infrastructure protection andrecovery.” Infrastructure for electric power, communications, water, wastewater,transportation,oilandgaspipelines,andmanufacturingsupplychainsaresystemsofsystems.TheissueofINSIGHTaddressedthevulnerabilityofinfrastructuretohighimpactthreatssuchasspaceweather,electromagneticpulse,cyberattacks,andphysicalattacks.Theemphasis is tomissionengineerinfrastructureSoStoberesilienttosuchattacks.AustralianLand-ForceCapabilityIntegrationAustralian Land-Force Capability Integration addresses the system of systems integrationchallenges,alsocalledcross-project integration, facing theorganizationofAustraliandefenseforces. These challenges include the integration of interoperable communication, commandand control, and support systems between various platforms which link, for example, theLanding Helicopter Dock with air support, amphibious watercraft, support ships, and landforces,inordertoprovidetheoverarchingamphibiouscapability.Theobjectiveistorealizethenetworkedforcedescribed inthe2009DefenseWhitePaper: ‘DefendingAustralia intheAsiaPacificCentury:Force2030’,andalsoconsideringtheAustraliannetcentricwarfareroadmapand the ISR roadmap. The Australian program leverages the US DoD SoSE approach, but istailoredtoAustralia’sapproachtodefensematters.An importantrecognition is thatSoStestandevaluationhasfundamentallydifferentgoals,character,andintentthanstandalonesystemacquisitiontestandevaluation.UKRemotelyPilotedAircraft(RPA)OperationsThe UK term formission engineering is ‘capability engineering’. The UKMinistry of Defence(MOD)hassimilarexperiencesintheengineering,acquiring,andoperatingoftheirRPAsystemasdescribedbyMindell(2015)forUSRPAoperations.SimilartoUSexperiences,thechallengesare in the compositionand integrationofdisparate command, control, communications, andoperationsassetsconstrainedbyprotocolsandend-to-endlatencies.

2.1.4CRITICALAREASFORADDITIONALRESEARCHINMISSIONENGINEERINGIDENTIFIEDFROMOPENSOURCELITERATURE

TherewerethreeareasthattheRT-171teamidentifiedintheopensourceliteraturethatarecriticaltoachievingthedesiredcapabilitiesinthecontextofreal-worldoperations:

1. Non-determinism of real-world phenomena – the techniques and tools to performmissionanalysisandengineeringappeartobedeterministicinnature;therealworldis


24

quitetheopposite(DeiotteandGarrett2013)(Marvin,Whalen,Morantz,Deiotte,andGarrett2014).

2. Explicitly accounting for systems operational availability Ao < 1, where Ao is theoperationalavaiabliltyofamissionsystem, in realworldscenario this is rarely“1”. ArelevantexampleistheoperationalavailabilityoftheintegratedsystemofsystemsforPredator, Gray Eagle, and Reaper remotely piloted aircraft (RPA) operations (Mindell2015).

3. Explicitly accounting for the human operators and commanders in the loops of thesystemsof systems–humanbeings require time to sense, think, interact, anddecidethat impacts the theoretical performance of systems of systems. Again, a relevantexample is the PEST (political, economic, societal, and technological) factors oneffectivenessofRPAoperations(Mindell2015).Anon-defenseexampleisthelandingofUSAirwaysFlight1549intheHudsonRiverbetweenNewYorkCityandNewJerseyonJanuary 15, 2009 after a bird strike resulted in the shutdown of both engines. Initialanalysis of the National Transportation Safety Board (NTSB) replaying the flight onsimulators indicated that the aircraft could havemade an emergency landing back atLaGuardiaAirportoratTeterboroAirportinNewJersey.Theflightsimulatorscenariosdidnotaccountforthetimefortheaircrewtoassesswhathappened,understandthestateoftheiraircraft,andregainsituationalawareness.Factoringinthelatencyoftheaircrew in the simulators gave the result that an emergency landing at LaGuardia orTeterborowasnotviable.

Missionengineeringisanemergingdisciplineandthelimitedavailabilityofreferencesintheseareas(items1-3above)seemstobeevidenceofthat.

2.2DEFININGMISSIONENGINEERING

AccordingtotheUSDepartmentofDefense(DoD),missionengineeringisdefinedas:

Aside from the defense domain, the definition of mission engineering expands to otherdomainsaswell.ButlerandWoody(2017)definedmissionengineeringasunderstandinganddocumenting end-to-end execution of a mission to understand how all the SoS parts worktogether.

In the “Mission Engineering Integration and Interoperability” article byMoreland (2015), heexpanded the definition of mission engineering as “planning, analyzing, organizing, andintegratingcurrentandemergingoperationalconceptsforthepurposeofevolvingtheend-to-end operational architecture and capability attributes, across the Doctrine, Organization,Training, Materiel, Leadership and Education, Personnel and Facilities (DOTMPLF) spectrum,including anticipatedBlue Force (BLUFOR) andOpposition Force (OPFOR)behaviors, that are

Thedeliberate planning, analyzing, organizing, and integrating of currentand emergingoperationalandsystemscapabilitiestoachievedesiredwarfightingmissioneffects.(Gold2016andDefenseAcquisitionGuide2017)


25

neededtoinformthecommunitiesofinterestinvolvedinfulfillingmissionneedsstatements”.Moreland (2015) described mission engineering as “a systematic, quantifiable, and iterativeapproach for assessments of naval technologies, systems and capabilities with a system-of-systems (SoS) approach that combines the structure of Systems Engineering and the tacticalinsightsofoperationalplanning”.

On December 9, 2010, the Chief of Naval Operation’s (CNO Integration and Interoperability(I&I)Summitwasbriefedonanewcollaborativeapproachtoprovideproactivesequentialstepsfor I&I activities for functional end-to-end accountability. This new approach incorporatesfinding the gaps in current capabilities (Warfare Capability Baseline), developing solutionrecommendations to find the operational shortfalls (Capability Solution Management), andprocessingtheresultsforapproval,execution,andimplementationwithintheNavy(Moreland,2015).Figure2illustratesthenewapproachforI&IsequentialactivitiesthatassociatesmissionengineeringandtraditionalSEwithinthe‘V’modelastimeevolves.

Figure2:MissionEngineeringwithintheSystemsEngineering‘V’Model(Moreland2015)

ItisimportanttodifferentiatethedefinitionofmissionforNASAfromtheUSDoD.AccordingtotheNASASystemsEngineeringHandbook,amissionisdefinedas“amajoractivityrequiredtoaccomplish anAgency goal or to effectively pursue a scientific, technological, or engineeringopportunitydirectlyrelatedtoanAgencygoal.Missionneedsareindependentofanyparticularsystem or technological solution” (NASA, 2016). The NASAMission Engineering and SystemsAnalysisDivision(MESA)atGoddardSpaceFlightCenterusesthetermmissionengineeringintheapplicationofengineering forspacemissions.TheNASAMESADivision“providesend-to-end mission systems engineering and guidance, navigation, and control capabilities andtechnologydevelopmenttoconceive,design,analyze,implement,verifyandvalidateonorbit,


26

and to support advanced scientific instruments and support platforms for ground-based,suborbital,andorbitalscienceandexplorationmissions”(NASA2017).NASArequiresamissionengineeringteam(i.e.operations,spacesegment,scientist,andgroundsystemsengineer)thatuses the mission engineering process to prioritize the mission and science operations in astructuredformintheearlydevelopmentphase.AccordingtoanearlierdefinitionofmissionengineeringbyNASA,missionengineering is “fundamentally aprocess improvement conceptthat addresses the interactions between the main mission elements of space, ground,operations,andscience”(OndrusandFatig1993).

Missionengineeringisdifferentfromsystemsengineeringbecausetheindividualsystemsthatare part of the military capability, such as ships and aircrafts, are inherently flexible,functionallyoverlapping,multi-missionplatformsthataresupportedbyacomplexbackboneofinformationcommunicationnetworks.Severalotheralliednationswerefoundalsotodescribemission engineering as capabilities engineering or force design. Capabilities engineering isexplainedintheLiteratureReview(AppendixD).

2.2.1MISSIONANALYSIS

Missionanalysis isperformed to “understandaproblemoropportunity, analyze the solutionspace,andinitiatethelifecycleofapotentialsolutionthatcouldanswertheproblemortakeadvantageofanopportunity”(SEBoK2017).

Missionengineeringdiffersfrommissionanalysisinthatthelatteronlyaddressesexaminationofcurrentoperationalandsystemcapabilitiesandnotthedesignandengineeringtoassurethemission.Inreferencetomissionanalysis,amodifiedresponsetoBlanchardandFabrycky(2011)definemissionasan“examinationanddefinitionoftheprimaryandsecondarypurposesofasystem”.

2.2.2CAPABILITYENGINEERING

The termcapabilityis widely used across many industrial sectors and has begun to take onvarious specific meanings across, and even within, those sectors. Terms such as capability-based acquisition, capability engineering and management, life capability management,capability sponsor, etc. arenowubiquitous indefenseandelsewhere.Henshawet al. (2011)haveidentifiedatleasteightworldviewsofcapabilityandcapabilityengineeringandconcludedthat the task of capability engineering is not consistently defined across the differentcommunities.Whilstmostpractitionersrecognizethatthere isastrongrelationshipbetweencapabilityandsystemofsystems(SoS),thereisnoagreeduponposition.However,therearetwobeliefsthatarewidelyacceptedamongthedifferentcommunities,including:


27

• a capability comprises a range of systems, processes, people, information andorganizations. (i.e. a system at levels three through five in Hitchin's (2003) five layermodel,suchasaCarrier-Strikecapability)and

• the capability is a property of the SoS (i.e. the capability of Carrier-Strike to engagetargetswithin300milesofthesea),whichmaybeemergentordesigned.

2.2.3SERVICESVIEWOFSYSTEMOFSYSTEMSENGINEERING

TheGuide to the Systems Engineering Body of Knowledge (SEBoK) has the following to sayaboutservice-orientedsystemsofsystems(BKCASEAuthors2017):

“A system of systems (SoS) is typically approached from the viewpoint ofbringing together multiple systems to provide broader capability. ThenetworkingoftheconstituentsystemsinaSoSisoftenakeypartofanSoS.Insome circumstances, the entire content of a SoS is information and the SoSbringstogethermultiple informationsystemstosupportthe informationneedsofabroadercommunity.Theseinformationtechnology(IT)-basedSoS’shavethesamesetofcharacteristicsofotherSoS’sandfacemanyofthesamechallengessuch as constituent systems havin their own purposes, priorities, and goals;possibly being outside the scope of control of the SoS, requiring reliance oninfluence;andasynchronyduetodiffernetlifecyclesofeachconstituentsystem.Currently, IT has adopteda ‘services’ viewof this typeof SoS and increasinglyapplies a International Organization for Standardization (ISO) 20000 series(Information technology -- Service management) or Information TechnologyInfrastructure Library (ITIL) v. 3 (OGC2009) based approach to thedesign andmanagementofinformation-basedSoS.AserviceperspectivesimplifiesSoSEasit:

• isamorenaturalwayforuserstointeractwithandunderstandaSoS,• allowsdesignerstodesignspecificservicestomeetdefinedperformance

andeffectivenesstargets,and• enablesspecificservicelevelstobetestedandmonitoredthroughlife.• Althoughithasnotbeenproventobeuniversallyapplicable,theservices

viewworkswellinbothITandtransportationSoS.”

2.2.4INTERVIEWRESULTSONTHEDEFINITIONOFMISSIONENGINEERINGBYINDIVIDUALPARTICIPANTS

The RT-171 team’s qualitative analysis is derived from 32 interviews with subject matterexperts and practitionerswho are involvedwithmission engineeringwork. Each interview isconducted in no more than 60 minutes and the questions are categorized in three mainsections, namely mission engineering, competencies, and future directions. The sample ofinterviewquestions canbe found inAppendix C. The first sectionof the interviewquestions


28

beginsbyrequestingtheparticipantstoprovidetheirdefinitionofmissionengineeringandalsotheir organization’s philosophy, processes, and approaches in mission engineering. Thefollowing figures provide the aggregated findings based on the interview responses and thesubtitleshowstheinterviewquestionthatcorrespondstoeachanalysis.Onthecharts,theredbarrepresentsthepercentageof intervieweesandthebluebarrepresentsthepercentageofexcerptsbasedontheinterviewtranscripts.Figure3showsthecategorizationofparticipants’variedresponsesonthedefinitionofmissionengineeringwith a percentage of excerpts based on the percentage of interviewees. All theinterviewparticipantsrespondedtothisquestion.Noneoftheinterviewparticipantssaidthatmissionengineering is adifferentdiscipline fromsystemsengineering, although twoof themdescribedsomeofthedifferencesbetweenmissionengineeringandsystemsengineering.Thevariationsinresponsesorderedfrommostfrequenttoleastfrequentareasfollows:

• Missionengineeringissystemsengineeringincludingsomeadditionalactivities(referredtoas“MEisSE+”inFigure3,

• Missionengineeringissystem(s)ofsystemsengineering,• Missionengineeringissystemsengineering,and• Therearedifferencesbetweenmissionengineeringandsystemengineering.

Figure3:PercentageofExcerptsandIntervieweesBasedontheirDefinitionsofMissionEngineering

Basedontheinterviewresponses,thefollowingaresomedirectquotationsontheirdefinitionsofmissionengineering.Acoupleoftheinterviewexcerptsfor“MEisSE+”arecitedas:

75%72%

34%

6%

54%

44%

24%

3%0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

ME is SE+ ME is SoSE ME is SE ME vs SE Differences

Defining Mission Engineering (N=32)(In your own words, what is mission engineering?)

Percent of Interviewees Percent of Excerpts


29

• “Ittakesholisticthinkingonestepfurther.InSE,partsareimportantasitrelatestothewhole.TocompleteamissioncanonlybeaccomplishedthroughextensionorscalingofSE.”

• “Mydefinitionisbasedonmymissionbasedonthecapabilitytosupportspecificmissions.WanttomakesureIhavefullend-to-endsystemsengineeringdisciplinestoengineer,design,test,andvalidateforthatcapability.”

Someoftheinterviewresponsesfor“MEisSoSE”areasfollows:

• “IviewMEastheabilitytounderstandacollectivesetofsystemsandtreatthemasSoS,howtheyrelatetoandinterrelatetooneanotherandhowtheycanbemodeledanddesigned.”

• “Makesuremissionessentialfunctions(missionthreads,missioncriticalfunctions)areaccomplishedacrossallsystems,includingSoS,andinmanycasesoperationsfunction.”

Someoftheinterviewexcerptsfor“MEisSE”areasfollows:

• “Missionengineering(ME)isSEasappliedtodesignandtheintegrationofkillchains.”

• “MEisSEasappliedtosolvingthemission.”

Theinterviewresponsefor“MEvsSEDifferences”isasfollows:

• “Sometimes,SEmaymissthemarkbecauseitdidn’thavetherelationshipwiththeoperation.”

Figure4showsthepercentagecoverageontheoveralldefinitionofmissionengineeringcodingbytheorganizationorservicebasedontheNVIVOqualitativeanalysissoftware.BoththeNavyandArmyprovided themost coveragecompared tootherorganizations,which is reasonablegiventhemajority(66%)oftheinterviewparticipantswerefromtheServices.


30

Figure4:PercentageofExcerptsontheDefinitionofMissionEngineeringbyOrganizationType

Whilethemostfrequent interviewresponsesconsideredmissionengineeringasanextensionofSE,Figure5depictsthecodingbyorganizationorservice.Theinterviewresponsesfromallthe participating organizations have a general understanding that mission engineering is anextensionofsystemsengineeringandconsideredasSoSE.Figure5indicatesthattheNavyhasaclear understanding on the definition ofmission engineering, and could be attributed to theearlydevelopmentalworkonmissionengineeringI&IbytheNavy.

45%

17%

12% 10%

4% 3% 3%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Navy Army DoD (Notservice specific)

Government(non-DoD)

Air Force Commercial/Industry

ProfessionalSociety

Perc

enta

ge o

f Exc

erpt

s

Organization Type

Excerpts Percentage on the Philosophy of Mission Engineering by Organization Type (N=32)

(What is your organization's philosophy on mission engineering?)


31

Figure5:InterviewResponse:“MEisSE+”CodingbyOrganizationType

The interview responses for theparticipants’ definitionofmissionengineeringare comparedwiththedefinitionsofmissionengineeringthatwefoundfromourliteraturereview.MorethanhalfoftheparticipantsdefinedmissionengineeringasanextensionofSEandconsidereditasSoSE,whichalignswiththedefinitionsofmissionengineeringprovidedbytheUSDoD,Navy,NASA, and other sources. None of the participants mentioned that mission engineering iscompletelydifferentthanSE,althoughoneoftheparticipantsmentioneddifferencesbetweenmission engineering and SE. There is a clear link in the data between SE and missionengineering. There is not universal agreement on how the two are related and interviewresponsesaboutthedistinctionsaresituational.

2.2.5INTERVIEWRESULTSONTHEORGANIZATIONDEFINITIONANDPHILOSOPHYONMISSION

ENGINEERINGBYINDIVIDUALPARTICIPANTS

The participants in this study revealed their perspectives on their respective organization’sdefinitionandphilosophyonmissionengineering.Figure6depicts the interviewparticipants’perceptions and understanding of their organizations’ definition and philosophy on missionengineering,whichcanbesummarizedas:

• A comparison between the interviewees own definitions (Figure 3) with theirorganizations’ definitions of “mission engineering is system of systems engineering”indicatesthattheirphilosophyisalignedwiththeirorganizations

50%

20%

14%12%

4%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Navy Army Government(non-DoD)

DoD (Not servicespecific)

Commercial/Industry

Perc

ent o

f Exc

erpt

s

Organization Type

"ME is SE+" Definition by Organization Type (N=32)


32

• The participants considered their organizations’ understanding of the operationalcontextasthemainphilosophyonmissionengineering

• Fifteen percent of them said their organizations have no definition or philosophy,followedbysomethat indicatedtheirorganizationsgenerateminimumeffort tomeetmissionobjectives

• A minimal amount of interview participants perceived their organizations as beingconfusedwith thedifferentdefinitions,and someevenconsideredMEasoutof theirorganizations’scope

Figure6:PerspectivesontheOrganizationDefinitionandPhilosophyonMissionEngineering

Based on the interview question “What is your organization’s philosophy on missionengineering?”, the following interview excerpts were extracted and some were redacted toprotecttheparticipants’privacyandconfidentiality,asshowninTable3.

45%

24%

17% 17%

10%

3% 3%

22%

12%8% 8%

5%2% 2%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

ME is SoSE UnderstandOperational

Context

CapabilityDevelopment

No Definition orPhilosophy

Minimum Effortto Meet Mission

Objectives

Confusion withDifferent

Definitions

ME is Out ofScope

Organization Definition and Philosophy on Mission Engineering (N=32)(What is your organization's philosophy on mission engineering?)



33

Table3:SelectedInterviewExcerptsontheOrganization'sDefinitionandPhilosophyonMissionEngineeringMissionEngineering

Activities InterviewExcerpts

MEisSoSE

• “FromMEperspective,yesthereisaphilosophy,frommyorganization’sperspective.SowhatIdoissystemofsystemsengineeringandintegrationfortheorganization’snetworkandanytechnologyinsertionintotheorganization.”

Understandoperationalcontext

• “Usingsystemsengineering(SE)processeswithadifferentperspective.ApplySEprocessesintwodifferentperspectives–inplatformandlookingmoreholisticallyacrossorganizations.Theprocesseswereapplicableinestablishingconceptofoperations(CONOPS)forcustomersandlookingatthemtodetermineCONOPStobeeffective,andtheplatformsneededtoperformthem.”

Capabilitydevelopment

• “Webreakdowntheprocess.Whenwe’resupportingmissiongaps,wetake that capability gap; all stakeholders are identified, what therequirements are, broken down into stakeholders and informationexchangerequirementsandhowtheinformationexchangesaredone.”

• “Missionsystemsengineering istheanalysistounderstandthenetworkorcapabilityyou’redesigningwillenhancethesoldiers’mission.Youcanbuild all the arch you want but they are only pictures. Analysis willinform you. Analysis early enough. So much new technology – havedifferent levels of fidelity of analysis. Youmight have to gowith lowerfidelitymodelsforthat.That’sthemostimportant/criticalpiece.”

• “Doing mission analysis and business analysis are all part of it. We’redriven to do mission and business analyses in a set of processes inINCOSEandISO.”

Nodefinitionorphilosophy

• “I’mnotsureifwehaveaphilosophy.”

• “Idon’tthinkI’veheardfromthem.”

Minimumefforttomeetmissionobjectives

• “Minimumrequirementstomeetthemissionobjectives.”

• “Strugglingwiththatbutlookingattheoperationalcontext,COCOM(CombatantCommand),givingmorecontexttowherethemissionisheadingto.Haven’ttrickleddowntothesystemandtechnologieslevels.Strugglingtoreachconsensusandtransient.Noonewantstochampionandtakeownership.”

Confusionwithdifferentdefinitions

• “Alwaysbeenconfusing.Alotofpeoplehavedifferentdefinitions.First,thisisnotclearlyunderstoodbytheworkforce,programoffices,andasaresult,don’tgetfunding.”

MEisoutofscope• “Missionengineeringisconstrainedwithintheboundssetbyaprogram.

Thescopeoftheprogrammaynottakeamission-orientedapproach.Andtheprogramprovidesthemoney.”


34

2.2.6RESEARCHFINDINGSONTHESCOPEOFMISSIONENGINEERING

This sectionbeginsby reporting thecriticalactivitieswithin thescopeofmissionengineeringbasedontheinterviewparticipants’positions,keyresponsibilities,andkeyvalueprovided.Theparticipants provided insights that are crucial in the mission engineering activities. Figure 7showsthe interviewresponsesanalysison thecriticalactivities inmissionengineering,whichcanbesummarizedas:

• Criticalmission-focusactivitiesbeginfirstandforemost,withanunderstandingofthemission as the highest overall compared to other activities, indicated by the highestpercentageofinterviewparticipants

• Toptechnicalactivities includethearchitecture,analysis, requirements,modelingandsimulation, capability development, integration and interoperability, testing andevaluation, technical assessments, and composition – all of which are recognized asdifficultinacomplexmissionenvironment

• Othernon-technicalactivitiesincludecommunication,workforcedevelopment,anduncertaintywhendealingwithmissionengineeringwork

Figure7:MissionEngineeringCriticalActivities

56%

50%

38%

34%

28%

28%

22%

22%

22%

16%

9%

9%

9%

11%

11%

8%

7%

6%

6%

6%

4%

4%

3%

2%

2%

2%

0% 10% 20% 30% 40% 50% 60%

Understand the Mission

Architecture

Analysis

Requirements

Modeling & Simulation

Capability Development

Integration & Interoperability

Testing & Evaluation

Communication

Workforce Development

Technical Assessments

Uncertainty

Composition

Mission Engineering Activities (N=32)(Please describe your current position. What are your key responsibilities? What is the key value you provide in

this position? What is the most critical thing you do to be effective in your current position?)



35

To provide more detailed activities in mission engineering, selected interview excerpts arelistedinTable4.Someinterviewexcerptsareredactedtoprotecttheparticipants’privacyandconfidentiality.

Table4:SelectedInterviewExcerptsonCriticalActivitiesinMissionEngineeringMissionEngineering

Activities InterviewExcerpts

Understandthemission • “Understandthemission.Weneedpeoplethatunderstandthewarfightingenvironment.Weneedsomeonewhounderstandstherealityofthewar.”

Architecture • “ThemissionanalysisandmissionarchitecturecompetenciesareimportanttoperformingME.”

• “Architecture-centricanalysisprocessforlookingatthingsfrommissionengineeringpointofview.”

Analysis • “Webreakdowntheprocess.Whenwe’resupportingmissiongaps,wetakethatcapabilitygap;allstakeholdersareidentified,whattherequirementsare,brokendownintostakeholdersandinformationexchangerequirementsandhowtheinformationexchangesaredone.”

• “Missionsystemsengineeringistheanalysistounderstandthenetworkorcapabilityyou’redesigningwillenhancethesoldiers’mission.Youcanbuildallthearchyouwantbuttheyareonlypictures.Analysiswillinformyou.Analysisearlyenough.Somuchnewtechnology–havedifferentlevelsoffidelityofanalysis.Youmighthavetogowithlowerfidelitymodelsforthat.That’sthemostimportant/criticalpiece.”

• “Doingmissionanalysisandbusinessanalysisareallpartofit.We’redriventodomissionandbusinessanalysesinasetofprocessesinINCOSEandISO.”

Requirements • “Therequirementsflow-downandunderstandingwhat’srequiredtotranslatingtorequirementstomeettheneedsisacriticalpiece.”

• “TheintegrationwithSEandMEmeetattherequirements.SErequirementsmanagementandelicitationshouldbecriticaltowheretheymeet.”

Modeling&Simulation • “Agent-basedsimulationsintheoperationalcontext.Thereisopportunitytoputinthejointfleetexercisegroups.Wetakeoperationalthreadsandkill-chainandbuildanagentforeveryone,andbuildahigh-fidelitymodelwithbehaviors.Werunsimulationstocaptureemergingbehaviors.Itcanpredictwhatthingsshouldlooklike.”

• “WeuseModelBasedSystemsEngineering(MBSE),SysML,andotherengineeringtoolsonatimelinetohaveacompletefunctionalflowblockdiagram(FFBD)andconceptofoperations(CONOPS).Iterate


36

MissionEngineeringActivities InterviewExcerpts

untilallthefunctionsworktogether-functions,products,whattheyneedtodo,breakdownfunctions,understandtheinterfacesfromonefunctiontoanother.”

• “Emergenceofcomputationalsimulations–eraofbigdata.Bigdatainthecommercialsectorusuallyhaslotsofconsumerdatatouse-wedon’thavethatluxury.Butwe’rebuildingonthattogathermoredata.Sometimesdecisionmakersoversimplifybutreallytheyneedtoconsiderthefullhierarchy.”

CapabilityDevelopment • “Lookingattheoverallcapabilityofsystemsengineering.Thereissystemsengineeringandthereissystemofsystemsengineering(SoSE).Systemsengineeringisbuildingaweaponsystem.SoSEisputtingtheweaponsystemintothewholepicture.”

• “Weareinitiatingeffortstoplanfuturecapabilitiesandclosegapsatthemissiondomainlevel.”

Integration&Interoperability

• “Wetookthebitsandpiecesandthekitwehadandknitthemtogetherintosomethingusefulforthewarfighters.”

• “IthinkthebestsimplestdescriptionIhaveismissionengineeringisaboutconnectingtheLegopiecesinawaythatwillsupportaparticularmission.It’snotaboutdecidingwhethertogowithLegoorK’nexandit’snotaboutbuyingthepieces.It’stheendofthedevelopmentprocess.HowcanIbestsatisfymycustomer’sneedswithwhatIhave?”

• “Themostcriticalthingwedoistheintegrationofinformation.That’skeyforspearheadingthis.”

Testing&Evaluation • “Testingatlargersystemofsystems(SoS)toconfirmiftheycanmeettherequirements.”

• “Testmissionthreads.”

• “Testingtoolsetisessentialtoo.”

Communication • “Communicationespeciallywiththepartsandpiecesthatgoesintotesting(professionalandtechnical).Getrightinformationtotherightpeople.”

• “TrytoattendallthetechnologyreviewsthatIcantoprovidetheProgramofRecordswhethertheycanuseitornot,togivethemtheknowledgeiftheycanmovethingsforward.”

WorkforceDevelopment • “Iwouldliketoseeaprocessforthedevelopmentoftheoperationalskillset.SomethinglikearotationorinternshiptogetthemintotheFleetcommandand/orexercises,toseeallthethingsthatencompassesamission.Intermsofmanagement,engineering,andsystemsengineering,we’vegotalltheprogramsandcohortsfor


37

MissionEngineeringActivities InterviewExcerpts

operationalexperience.NotacourseforaverageacquisitionpersonnelbutforthoseintheMEpath,itwouldbeinvaluable.”

TechnicalAssessment • “Needtoensurealltherightplayersareinvolvedtoanalyzethefullsetoftechnologicalandoperationalsolutionssothattheanalysesanddecisionsare“fully-informed”atallsecuritylevels.”

• “Iamfocusedonthematerielsolution.Theoutcomeismeetingthetechnicalperformance.”

Uncertainty • “BeingabletopredictthefutureandwhatitisthatyouwantsoyoucanclearlyIDthemissionsystemsengineeringthatyouneedtosupportthat.Wemightknowsomethingfor3yearsoutbutbythetimewegetthere,it’schanged.Constant,evolvingunknowns.”

Composition • “Oneofthethingswedevelopedwasaseriesofwayssomethingcouldfailandpatternsforhowtodesignaservice.Wedevelopedpatternsforcomposition.”

• “Becausewebelieveintheuniversalityofthesystem,wetreatitasmissioneffectiveness,trueboundingatthelevel.”

The interview data also points to the lack of factoring non-determinism in current missionengineeringwork, as shown in the literature reviewanalysis and follow-upsonprivate,deepbackgroundconversations.Sincemissionengineeringworkisrelativelynew,thequestionaboutthe mission engineering processes and practices were posed to the interview participants.Figure8showstheanalysisoftheinterviewresponsestothequestionontheirorganizations’currentprocessesandpracticesinmissionengineering,whichcanbesummarizedas:

• Understandingtheoperationalcontextwasdeemedasthemostimportantstepintheprocesses and practices inmission engineering, followed by an understanding of thecapability and adherence to military standards in order to identify gaps in theircapabilitiesoftheoverallmission

• The SE technical processes such as architecture, modeling and simulation,interoperability, and feasibility analysis were applied inmission engineering, within acontinuousfeedback

• Thenon-technicalpracticesandprocesseswerediscussedwhenterm“coalitionofthewilling”wasinvokedtwiceasbeingapartofthemissionengineeringpractices,aswellastheacquisitionprocessasbeinginfluentialinmissionengineering.


38

Figure8:ProcessesandPracticesinMissionEngineering

Table5providessomeinterviewexcerptsbasedonthequestion“What isyourorganization’sprocess or approach to performing mission engineering?”. Some interview excerpts areredactedtoprotecttheparticipants’privacyandconfidentiality.

Table5:InterviewExcerptsonProcessesandPracticesinME

42%

35%

35%

19%

10%

6%

6%

6%

6%

6%

6%

12%

12%

12%

8%

3%

3%

3%

3%

2%

2%

2%

0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%

Understand Operational Context

Architecture

Understand Capability

Identify Gaps

Modeling & Simulation

Acquisition

Standards

Interoperability

Coalition of the Wiling

Continous Feedback

Feasibility Analysis

Processes and Practices in Mission Engineering (N=32)(What is your organization's process or approach to performing mission engineering?)


ProcessesandPracticesinME SelectedInterviewExcerpts

Understandoperationalcontext

• “WorkwithFleettounderstandtheiroperationalneedsandtranslatethemintorequirementsthatthetechnicalcommunitycanunderstand.”

• “Alotofthiswouldbesimplestepsofprocessesandrules–combinewithsystemsengineeringtododesignactivitiesforprocessesandtechnologiestothinkofarchitecturedesign,managementprocesses,operationalcustomerneeds,businessneedsperspective.”


39


Architecture

• “Theoperationalarchitectureincludesallmissiontypesoffunctionsandthemissionarchitectureconnectswiththesystemarchitectureandoperationalarchitecture.”

• “Architecture-drivenanalysisisaprocessthat’sbeenongoinginourorganization.”

• “Wedeveloparchitectureproductstounderstandthesystemandhowtoemploycapabilitiesinanexercisewherethere’sholesorgaps.”

Understandcapability

• “Wetailorahighlevelsystemsengineeringmissionthatstartswithbeingabletounderstandtheroadmapofwhereourorganizationneedstogoandgooutasfaraswecanwherewehaveclarityofon.Workacrosscommunitytounderstandcapabilityandabilitytounderstandcapabilitybycertaintimeline.”

• “Mapoutspaceandlookatcapabilitytolookforgapsandmakeadecisionifweneedtochangeanything.”

Identifygaps

• “Identifygapsincapabilityandchange.Inthelongterm,Imaylookatnewtechnologies,andhowitcanimpactandclosegaps.Itmaynotbetiedtoanyprograms.”

• “Theproblemwithmissionneedsandprogramsis,howdoyouclosethegaps?Thereisnoadministrativecontrol.”

Modeling&simulation

• “Usingamodel-basedapproachtodefineandmanagetheprocessend-to-end.TheyusearchitectureasaframeworkintheNavytomanagethemissionthreadbutalsoasasystemsmodeltodefinerequirements,definiteanalyses,andmanagebehaviorsthroughoutthelifecycle.”

Acquisition• “Wecollectandvalidatethedataandseeiftheinstrumentisgood

enoughforthegovernmenttopurchase.”

Standards

• “Itismorethanmilitarystandards,e.g.electromagneticpulse(EMP)standards,butmilitarystandardsarethereforsure.Forexample:protection(documentationrequirements)-useascorerequirementsthatneedtobeadheredto.”

Interoperability

• “Bringingtogetherdiversedomains(undersea,land,space,etc.)tobeintegratedtomeettheneedsofthemissionthroughjointoperationsplatforms,keytechnologiesandriskmaturitytodeterminetheoperationalneeds.”

Coalitionofthewilling

• “Someprogramsseethebenefitupfront(coalitionofthewilling).”

• “Forgapsinstandards,clarifyingthesestandardscanbedifficultandmayimpactthebaseline.“Coalitionofwill”insuresystemsisfulfillingitsroleinSoS.”


40

To enrich themission engineering body of knowledge, 31 of the participantswere asked todetermine if there was any overlap between mission engineering and SE based on theiractivities. Figure 9 displays the results where an overwhelming majority of the participantsindicatedthatthereisoverlapbetweenMEandSEwork.OnlyoneoftheparticipantssaidthatthereisnooverlapbetweenmissionengineeringandSEintheindividual’sworkexperience.

Figure9:InterviewResponsesontheOverlapofMissionEngineeringandSystemsEngineering

Figure 10 shows the interview responses for those who indicated that there is an overlapbetween mission engineering and SE to better understand what they are. The majority


Continuousfeedback

• “AllowsmetoadjustarchitecturebasedonfeedbackandwecontinuethatprocesssowegettothecapabilitythattheArmyapprovesthenworkthetailendofhowtofieldthatcapability.”

• “Continuousfeedbackloop.”

Feasibilityanalysis

• “Lookatalternatecourseofactionstoaddressthecapabilitygap,toincludefeasibilityanalysisbasedonthestakeholder’sfeedbackandfeedingbacktotheorganization.”


41

considered mission engineering as a form of SoSE. In regards to this question, the otherresponsesinclude:

• Missionengineeringandsystemsengineeringareinterchangeable,• Missionengineeringisasubsetofsystemsengineering,and• Missionengineeringisanextensionofsystemsengineering.

Figure10:InterviewResponsesfromthosewhoindicatethatthereisOverlapbetweenMissionEngineeringand

SystemsEngineeringWorkTable6providessomeinterviewexcerptsbasedonthequestion“Doyouseesomeoverlapintheactivitiesof systemsengineeringandmissionengineering?”.Some interviewexcerptsareredactedtoprotecttheparticipants’privacyandconfidentiality.

Table6:SelectedInterviewExcerptsontheOverlapbetweenSystemsEngineeringandMissionEngineering

50%

30%

23%

13%

24%

14%11%

6%

0%

10%

20%

30%

40%

50%

60%

ME is a form of SoSE ME and SE are Interchangeable ME is a Subset of SE ME is SE+

Overlap between Mission Engineering and Systems Engineering Work (N=32)(Do you see any overlap in the activities of systems engineering and mission engineering?)


OverlapbetweenSEandME SelectedInterviewExcerpts

MEisaformofSoSE

• “Missionengineeringisaformofsystemofsystemsengineering(SoSE).Take systems engineering practices and apply to morecomplex problems. When you view the system with the builtcomponents,thensystemsengineeringpracticesmakessense.”

MEandSEareinterchangeable

• “IthinkMEisSEforme.Ithinkasystemcanbedefinedasapart,system,weaponsystem,SoSorhow it’sdefinedasamission isthe system. It’s a different perspective but the fundamentalpreceptsofSEstillapply–it’saperspectiveofhowyouviewataglobalor10,000feetlevel–missionasafocus.”


42

2.3ACADEMICPROGRAMINMISSIONENGINEERING

According to the Worldwide Directory of Systems Engineering and Industrial Engineeringdeveloped by the Systems Engineering Research Center (SERC) at Stevens Institute ofTechnology and the International Council on Systems Engineering (INCOSE), there are 112universities worldwide offering systems engineering academic programs (SERC, 2017). Therearesomeuniversitiesthatprovidesystemofsystems(SoS)curriculum(refertoAppendixE).

Based on our research, there is only one established mission engineering program. Thisacademic program is a graduate certificate program fromOld Dominion University (ODU) inMissionAnalysisandEngineering.AccordingtothisODUgraduatecertificateprogram,missionengineering is “an emerging discipline in which system-of-systems engineering tools andpractices are combined with the tactical insights of operational planning” (ODU, 2017). TheODUprogramisdesignedwithafocusontheU.S.Navy’soperationalneedsandrequirements.Thelearningobjectivesinclude“integratingtheFleet,Technology,andAcquisitioncommunitiestodevelopadvancesinwarfightingcapabilitiesacrossmissionareas”(ODU,2017).

ThecloseproximitybetweentheODU-DahlgrenandtheNavalSurfaceWarfareCenter(NSWC)DahlgrenDivision isgearedforthe localcommunitiesof interests,andallowspractitionerstoparticipate intheODUMissionAnalysisandEngineeringgraduatecertificateprogram.Oneofthe ODU adjunct professors who developed the ODU mission engineering curriculum andteaches one of the courses is Dr. JamesMoreland, SES, Deputy Director for NavalWarfare,Office of the Secretary of Defense Acquisition, Technology and Logistics. The ODU missionengineeringcurriculumcanbe found inAppendixF.Thestudents take fourcourses includingone required course, three electives, and a one-credit capstone course where the studentsapplytheknowledgefromthefourcoursesthroughproject-basedlearning.

MassachusettsInstituteofTechnology(MIT)startedashortcourseinthesummerof2017thatisrelatedtomissionengineeringtitled,“SurfaceShipCombatSystemDesignandIntegration”.This course is classified SECRET/NORFORN, and is open to active-duty U.S. military, U.S.government employees, and U.S. civilian contractor personnel with U.S. governmentsponsorship.Acceptedstudentsareexpectedtohavematuretechnicalbackgroundsbasedontheirexperienceoreducation(equivalenttoagraduateeducation).Theone-weekcoursewasalsodevelopedbyDr.JamesMorelandwithanobjectiveofprovidingstudentswithknowledgeofsurfaceshipcombatsystemsandthefactorsthatimpacttheirintegrationamongthemselvesandaboardship,aswellasthe impactofmissionsandthreatsastheyrelatetoplatformandsystemdesignconsiderations.TheMITcoursecoverstopicssuchas:

OverlapbetweenSEandME SelectedInterviewExcerpts

MEisasubsetofSE

• “Absolutely. Mission engineering is yet another bucket in themultitude of systems engineering. Systems engineering is waytoo broad of a definition already but mission engineering fitswithinit.”

MEisanextensionofSE(MEisSE+)

• “MEisbiggerandbroaderthanSE.SEisaboutthesolution.MEisabouttheoperationalrequirements,solutions,andcapability.”


43

• IntroductionandOverviewofCombatSystemArchitecture,

• MissionandRequirements,

• ArchitectureforShipsandCombatSystems,

• DiscussionofSpecificWarfareAreas:SurfaceandLandAttack,Aegisandnon-Aegis(AAW),andBallisticMissileDefense,

• IntegratingintoShipArchitecturesandImpact,

• IntegratedTopsideDesign,

• AdvancedTechnologies,and

• IntegrationChallenges.

2.4RESEARCHFINDINGSONTHEGAPSINMISSIONENGINEERING

Whenthe interviewparticipantswereaskedabout theirperspectivesoncriticalchallenges inmissionengineering,thesechallengeswereidentifiedasgapsinthisstudy.Figure11showsthesummary of the aggregated analysis on the critical challenges to better understanding thescopeandcontextofmissionengineering.Toprovideadetailedview,theinterviewresponseswerecategorizedintotechnicalandnon-technicalchallengesinmissionengineering.

Figure11:SummaryoftheInterviewResponsesontheCriticalChallengesinMissionEngineering(N=32)


44

Figure 12 shows the interview responses analysis of critical non-technical implementationchallenges in mission engineering. The top challenges are all related to non-technicalimplementation issues related to workforce, culture, funding, governance, acquisition, andcommunications,andlackofinterestinmissionengineering,withworkforcebeingthehighestoverallchallenge.

Figure12:CriticalNon-TechnicalImplementationChallengesinMissionEngineering

Some of the interview excerpts are selected to highlight the critical non-technicalimplementationchallengesinmissionengineering,asshowninTable7.

Table7:SelectedInterviewExcerptsonCriticalNon-TechnicalImplementationChallengesinMENon-TechnicalImplementation

ChallengesSelectedInterviewExcerpts

Workforce

• “Iwouldsayworkforcedevelopment.Evenifyougetfunding,thereareveryfewpeoplewithexpertisetodothework.Thereisonlyahandful.”

• “The skillset we need is what everyonewants. I don’t know if we canattract them all. Competition in industry for that talent. Bring on folkswithanalyticsskills–almosteveryindustryisdoingit.”

• “Since most have 25 to 30 years of experience in large systems, weessentiallyhandpickedthem.Theproblem iswedon’thave theupandcomingengineerstoassumetheroles.”

44%41%

34% 34%31% 31%

16%

20%

8% 9%7% 8% 8%

3%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Workforce Cultural Funding Governance Acquisition Communication Lack of Interest inME

Non-Technical Implementation Challenges in Mission Engineering (N=32)(What do you see as critical challenges in mission engineering?)



45

Non-TechnicalImplementation

ChallengesSelectedInterviewExcerpts

Cultural

• “The#1wouldbe culturebecausewehave an industry of peoplewhohavebeendoingthingsthesamewayforalongtime.Gettingpeopletoembracechange.”

• “Providemoreopportunities to self-assessanddepartment investing inmoremissionenvironments.Unfortunatelyfundingisn’tapproachedlikethisrightnow–needaculturechange.”

Funding

• “Youwill never (with the current acquisition process) get SoSE fundedbecausethephraseis“wefundsystems,notSoSE”.EvenifMEbubblesup, itwillbeincrediblyhardtofindtheseproducts.Weknowtheyonlyfundsystems.IhaveneverseenfundingforSoSEcapability.”

• “Integratedcapabilitylist–butverybroadandgeneralbecauseit’sbeenunderfunded.”

Governance

• “In the implementation context by the U.S. DoD, the issue is bydefinition,theDoDisstovepipedintoservicesthathavegrownupintoservices that have gone from capability in battle spaces and fragmentsthem. It fragments the stakeholders and governance. The fundamentalissueiswetrytooverlapitinagovernmentalphase.”

• “Governance–howdoyoudealwithit?”

Acquisition

• “Theacquisitionprocess isbrokenbecause thegovernmenthasahardtimecuttingacrossthings.Acquisitioncapability ismuchmoredifferent–policytouchesdifferentsystems.”

• “Get away from the stone-pipe nature of the acquisition process…Changetheacquisitionprocessandthewaytheygetcapability.”

Communication

• “Improve communication across the board. Keep communication withSeniorLeadersandcommunicatedowntopeopledowninthefieldtodotheirjobssotheyunderstandandreapthebenefitsprovidedbyscienceandtechnology.”

• “Communication and decision making when not having the rightinformation. Sometimes data gets skewed. That’s something that wasdealtwith.”

LackofinterestinME

• “SeniorlevelsareunclearifMEisanoldornewdiscipline,ofifthesearejustbuzzwords.Thedangerorpotentialdetractor(ex.programoffice)isthe belief that it’smateriel instead ofmaybe change in tactics. Peoplearenotbuyingintothegeneralphilosophy.”

• “Mission support inME– the fact that youhave tohaveorneed jointbuy-in that theprocesswillhelp themat the service level,not just thejoint level. Our real intentions is not taking away any capability, butrather looking from a mission success standpoint. It is critical to haveeveryone’sbuy-in.”


46

Figure 13 shows the interview responses analysis of critical technical implementationchallenges inmission engineering. The top technical challenges include those related to theoperational context, requirementsmanagement, and testing– all ofwhichare recognizedasdifficultinasystemsorientedacquisitionsystem.

Figure13:CriticalTechnicalChallengesinMissionEngineeringTable8 lists someof the interviewexcerpts tohighlight thecritical technical implementationchallengesinmissionengineering.

Table8:SelectedInterviewExcerptsonCriticalTechnicalImplementationChallengesinME

25%

25%

25%

22%

22%

16%

16%

13%

9%

9%

6%

6%

5%

5%

5%

3%

3%

2%

2%

2%

0% 5% 10% 15% 20% 25% 30%

Lack of Operational Context

Requirements Management

Complexity

Integration & Interoperability

Testing & Evaluation

Analysis

Planning

Architecture

Implementation

Uncertainty

Top Technical Challenges in Mission Engineering (N=32)(What do you see as critical challenges in mission engineering?)


CriticalTechnicalImplementationChallenges

inMESelectedInterviewExcerpts

Lackofoperationalcontext

• “GettingengineerstohavethemissionofME.Somepeopleworkatasystemforalongtimebutdonotseethewhole.”

• “Ibelievethatmostpeoplearesystemsthinker.It’sjusttheirboundaryisnotaslargeasthebiggersystem.Idon’tbelievethatthereareenoughpeoplewhounderstandthebroadercontext.”


47



Requirementsmanagement

• “Missionengineeringwillhavealottolearnaboutgettingittomeettherequirements.”

• “Findingdocumentedfleet-levelrequirementsisachallenge.”

Complexity

• “Inourorganization,itissystemofsystemsengineering(SoSE)–emergingbehavior,independentbehaviors–haveallofit.Theyarealsochaoticsystemsthataremixedwithtraditionalsystems.MEhastotakeitallintoconsiderationwhereelsesystemsengineeringdoesn’t.”

• “ThebiggestproblemisthecomplexityoftheSoS.”

• “UseSEacrossallthedomainsandgetthemtotranslateacrossdomains.Thelevelofcomplexitygetsexponentiallychallenging.”

Testing&evaluation

• “Testingishardinmissionengineering.Eachprogramandtheirdetailedtestplanslookattheirindividualperformancerequirements.Whenyougotomissionrequirements,it’snottheresponsibilityofone,butthecollectionoftheprogram.Meetindividualrequirementsbutthere’samissionshortfall…Nooneisdoingmissiontesting.”

• “Testtowardsmissionrequirements.Whenyoudomissionengineering,itiscriticaltohaveprogramofficesparticipateinsolutionsdevelopments.”

Analysis

• “Missionanalysisismoreprecisewhendealingwithmathematicalmodel.SEdealswithmoreaccuratemodel.Ifyouhaveasystemsengineerdoingamissionarchitect,itwouldbebetter.”

• “Thereisalackofunderstandingthatthishasn’tbeendonebefore.Theydoanalysisonthreatsandkillchainsbutattheplatformlevel,notthemissionkillchaintobethemosteffectivetheorganizationcanbe.”

Integration&Interoperability

• “OnemorechallengethatisaconcernisrootedinSoS.ExtremetiesinSoSandME.Theabilitytogettherightdataattherighttimeattherightplace.”

• “MakeMEintegratedintheirwork.They’renotusedtoit.Everyoneshouldbeamissionengineer.Thatwouldbepartofthegeneralworkprocess.”


48



Architecture

• “InSEwetalkaboutphysicalarchitectureandfunctionalarchitecture.Butunderstandingthecommercialarchitectureisalsoimportant.IfIdon’tunderstandhowthecontractsandcompanyincentivesarestructured,Ican’tdoend-to-endservicedesign.Foralotofphysicalsystemswebought,Igottherelevantagreementinplace,butcouldhavedonebetterifweunderstoodhowimportantthecommercialarchitecturewas.”

• “Architecturedesignisessentialfromthatperspective.Withoutaproperarchitecture,youwon’tunderstandthesystem.”

Planning

• “Weareinitiatingeffortstoplanfuturecapabilitiesandclosegapsatthemissiondomainlevelintheorganization.Theorganizationandoversighttomakethisshifthappenisinprogress.”

• “Engineeringparameterwithconstraintsandscheduleonhowtoperformthemissionandstaywithinthecost–predictionandstaywithinthecost–it’ssomethingwestrugglewith.”


49

3:MISSIONENGINEERINGCOMPETENCY

Based on the data collected (Appendices C-E), the RT-171 team developed a competencyframework, which reflects the critical knowledge, skills, abilities, behaviors, and cognitions(KSABCs)formissionengineering.Thiswasdevelopedwiththreemainefforts:

• The literature review provided an understanding of the critical activities of missionengineering,whichprovidedinsightintotheskillsrequiredtoperformthoseskills.

• The interviews provided detailed insights into the KSABCs that practicing missionengineers believe are critical to their work and that have been critical to their ownsuccess.

• The team reviewed multiple existing systems engineering competency models toidentifycommonandconsistentwaystodefinesomeoftheoverlappingcompetencies.

3.1MISSIONENGINEERINGCOMPETENCYFRAMEWORK

The details of how the competency model was developed can be found in Appendix C:Methodology.

Figure14providesanoverviewofthecompetencyframeworkdevelopedbytheRT-171teamfor mission engineering. The framework was developed by developing critical competencygroupsfromthedatacollectedfrombothinterviewsandliteraturereview(seeAppendixDforadditional detail). The framework,which is foundedon the systemsengineering competencyframeworkpresentedinAtlas1.1(Hutchisonetal.2018),isorganizedinthefollowingways:

• Competency Areas are groupings of related knowledge, skills, abilities, behaviors,and/orcognition.

o EachCompetencyArea is comprisedofCategories,which are specific types ofknowledge,skills,abilities,behaviors,andcognitionwithsharedcharacteristics.

§ Some categories are further refined into Topics, as defined by thequalitativedataanalysis(AppendixD).

Figure 14 includes the Areas and Categories of the Mission Engineering CompetencyFramework.TherearesixAreas:

1. Discipline and Domain Foundations: This area focuses on the foundationalunderstandingofthesystemsthatwillberequiredtosupportagivenmission.

2. Mission Concept: This area focuses on an individual’s ability to understand andworkwithin the context of a given mission, including understanding the overall concept,scenarios,andrelevantmission threadsaswellasunderstanding the factors thatmayinfluencethemissioninadditiontotechnology(doctrine,processes,training,etc.).


50

3. SystemsEngineeringSkills:MissionEngineeringandSystemsEngineeringsharecriticaloverlaps (seeSection1).ThisareaprovidesclarityonthespecificsystemsengineeringKSABCsthataremostcriticalformissionengineering.

4. SystemsMindset:ThisareaisanalogouswiththesystemsmindsetinHelix(Hutchisonetal.2018)andincludesthecognitiveabilitiesaroundthinkingholisticallyaswellasbeingabletoidentifytherightlevelsofdetailandintegratetheseperspectives.

5. Interpersonal Skills: This area includes the skills and behaviors associated with theability to work effectively in amulti-team environment and to coordinate across themissionscope.

6. Technicalleadership:Skillsandbehaviorsassociatedwiththeabilitytoguideadiverseteamofexpertstowardaspecifictechnicalgoal.

Figure14.MissionEngineeringCompetencyFramework

EachoftheseAreasiselaboratedinthesectionsbelow.Theorderlistedabovedoesnotreflectpriorityor importance–allAreasare importantformissionengineering–butratherhelpstogroup similar items togetherand the secondArea generalbuildsupon theprevious.Viewingfromthetopinclockwiseorder,considerthefollowingclusters:

0

1

2

3

4

5

Discipline&Domain

Foundations

MissionConcept

SystemsEngineeringSkills

SystemsMindset

InterpersonalSkills

TechnicalLeadership

ExampleProfile1 ExampleProfile2

2.MissionConceptOperationalContext

MissionConceptofOperation

MissionScenarios/Threads

DOTMLPFSpace

6.TechnicalLeadershipGuidingDiverseStakeholders

TeamBuilding

PoliticalSavvy

DecisionMaking

WorkforceDevelopment

5.InterpersonalSkillsCommunication

Translation

EnterpriseContext

Building&UtilizingaSMENetwork

Coordination

Influence,Persuasion,&Negotiation

4.SystemsMindset‘BigPicture’Thinking

Adaptability

ParadoxicalMindset

Multi-ScaleAbstraction

CriticalThinking

3.SystemsEngineeringSkillsSystemofSystemsEngineering

Analysis

Architecture

ModelingandSimulation

Requirements

Integration

GapAnalysis

1.Discipline&DomainFoundations

PrincipleandRelevantDisciplines

RelevantDomains

SystemCharacteristics

RelevantSystems

RelevantTechnologies

AcquisitionContext


51

• What –Areas 1and2are focusedon thewhat–what is themission?What isbeingdeveloped?Whatdisciplinessupportthis?Whatarethecharacteristicsofthemission?Individualswith competencies in theseAreaswouldbeable toanswer these typesofquestions. The Discipline & Domain Foundations provide the crucial engineering andacquisitioncontext,while theMissionConcept buildson this knowledge todefine theexpectationsandcriticaloperationalcontextforthemission.

• How–Areas3and4arefocusedonhowmissionengineeringisperformed,withafocuson critical skills that enable individuals to performmission engineering. The SystemsEngineering Skillsprovide the engineering ability to domission engineeringwhile theSystemsMindset enables individual to apply the systems skills in thebroadermissioncontext.

• Who–Areas 5and6are focusedon thecritical skills toenabling the teams thatwilldeliver mission capabilities. Interpersonal Skills are critical for working in and acrossengineering teams while Technical Leadership skills are critical for leading andintegrating across the many teams engaged in building the systems that generate amissioncapability.

Notethatfoundationalskills–e.g.math,naturalorsocialsciences,generalengineeringskills-are not listed in the above. For all of the individuals interviewed for RT-171, a fundamentalgrounding in these areaswas assumed. In terms of career paths,most stated that they hadbeen systems engineers prior to becoming mission engineers, so this was considered anassumedskillsetamongmissionengineers.And,becausemissionengineersdonot frequentlyfunctionasengineersatasystemlevel,theymusthavesufficientskilltounderstandtheworkintheseareas,butarenotrequiredtoperformthisworkthemselves.

3.1.1AREA1:DISCIPLINEANDDOMAINFOUNDATIONS

Anygivenmissionwillcrossmultipledomainsandrequirethesupportandintegrationofmanyengineeringdisciplinestoberealized.Becausebasicunderstandingofmath,sciences,andthefundamentals of engineering are assumed, the foundational building block for missionengineering,then,isanindividual’sabilitiesinthecriticaldisciplines,domains,andtechnologiesforagivenmissionaswellasagraspofcomplexityandtheacquisitioncontext inwhichtheyoperate.ThisAreaprovidesagroundinginthecriticalsystemsthatwillenableamission.

1.1. Principle and Relevant Disciplines: Disciplines are fundamental areas ofeducation or expertise that are foundational to a system. For example, for acommunications system, electrical engineering will be an important discipline tounderstand,whilecivilengineeringwillbe lessrelevant.Specialtiesaredisciplinesthatsupport mission engineering by applying cross-cutting knowledge. Specialties includeReliability, Availability, andMaintainability (RAM), Human Systems Integration, SafetyEngineering, Affordability and other related topics. A mission engineer needs tounderstandwhichofthesedisciplinesaremostcriticaltosupportagivenmission.

1.2. RelevantDomains:Domain refers to the overarching area of application for a


52

given system; this includes things such as space, aerospace, marine, communication,finance, etc. Competency in relevant domains may enable an individual to be moreeffective.

1.3. SystemCharacteristics:Formissionsystems,severalspecificcharactersticswerelisted as critical and prevalent formission systems. Themost commonly-cited criticalcharacteristics are provided below. However, when using this framework, thecharacteristicsshouldbetailoredtoreflectthecharacteristicsofthemissionsystem.

a. Complexity: While systems will have a variety of characteristics that missionengineersmusthandle, formissionengineers, complexity is critical. Ingeneral,an individual will work on a spectrum of complexity, ranging from simple tocomplicated to complex to chaotic. (Adapted from the Cynefin framework,Snowden and Boone 2007) Complexity is generally note measured by thenumberofpartsofasystem–whichwouldbeameasureofhowcomplicatedasystem is – but of the interactions between system elements, disciplines, ortechnologies,andthepropertiesthatemergeoutoftheseinteractionsthatarenot present in the individual elements. Mission engineering involves multiplesystems and/or systems of systems, each with their own inherent complexityand,therefore,missionengineersconsistentlyworkinthe“complex”space.Onecategorizationofcomplexityincludesstructuralcomplexity,dynamiccomplexity,andsocio-politicalcomplexity (Collinsetal.2017);whileanother identifies twokinds of complexity: disorganized complexity and organized complexity (SEBoKauthors, “Complexity”, 2016). For mission engineering, this includes not thecomplexities of an individual system, but of the interactions betweenmultiplesystemsor systemsof systems thatwillenableagivenmission. Inmanyways,complexityisincreasedbythefollowingthreecharacteristics.

b. Uncertainty: Uncertainty is the result of not having accurate or sufficientknowledgeofasituation.(ISO/IEC2009)BecausemissionengineersaretryingtointegrateacrossavarietyofsystemsorSoS’s,itisnotpossibleforthemtohaveevery detail on each mission system. Because mission engineers also may befrom outside the organizations or even services where a system is beingdeveloped,itmayalsobedifficulttoobtainrequiredinformation.Beingabletofunction, and make decisions under uncertainty is a critical skill for missionengineers.

c. Asynchrony: Aswithmany SoS’s,mission systems have to deal with issues ofasynchrony: the quality or state of being out of concurrence in time. Whenindividualsystemsareacquired,theyeachhavetheirownlifecycle.WhenthesesystemsthenneedtobecombinedintoalargerSoStosupportamission,theiracquisition lifecycles may not change. As one interviewee stated, “But thechallengeisallthebitsaredeliveredasynchronously.If[acontractor]deliversanew system, they select technology thatwill bemature at CDR and then dealwith obsolescence once the system is in service. In that scenario, I wouldn’texpect my subsystems to change that often. But in mission engineering, you


53

havetoexpectthatdifferentbitswillbedeliveredatdifferenttimes.Some[are]nearly obsolete, some very new and untested.” To be successful, missionengineersmustbeabletonavigateasynchronoussystems.

d. Legacy Systems: Though this is not unique tomission systems, especially in adefense context is it critical to understandwhatmany legacy systemsmay beexpected to integrate into thebroadermission.One intervieweestated, “[We]do leap in technologybut [wearealso]dealingwith systems thatentered thefleet in 1952 or 1953 - need to account for these.” Another intervieweeprovided a specific example of this, “For example, aircrafts like the B-52 havebeen on the ‘on and off’ status for ongoing efforts. Due to funding cost orprogramofficeservice, itshiftedintomonitoringandweren’tactive–moreforknowledgemanagement.”Missionengineershavetobeabletodealwiththeseissues.

1.4. Relevant Systems in theMission Space: The two categories above define thesystems in the mission space and how they are expected to interact. This category,however, is focusedonthemissionengineer’sunderstandingofthesecriticalsystems.This is not to say that mission engineers must be experts in every system, but theyshouldunderstandthecontextofeachsystem,includingthemissionandorganizationalcontext inwhich it is being developed. This familiaritywill better enable themissionengineer to anticipate potential problems when integrating systems to perform aspecificmission.

1.5. Relevant Technologies: Within the context of a mission, there are specifictechnologies that are relevant. For example, on a marine system, these may betechnologiessuchasgasturbine,radar,andsonarsystems;andeachtechnologyhasitsownterminology,challenges,etc.Amissionengineermustbeawareofthemostcriticaltechnologiesforthesystemsthatareincludedwithinagivenmission.

1.6. Acquisition Context: The ways that systems are developed and acquiredprovides critical context and boundaries for the ways in which missions can beaddressed. Particularly, government acquisition systems have standard processes andrules that constrain the ways in which mission engineers can impact the systems ofsystems that must integrate to achieve a mission goal. Without understanding theconstraints of the acquisition system, amission engineer can not effectively enable agivenmission.Thisincludesunderstandingtheacquisitioncontextoftheprogramsthatsupport the mission and where each of these programs fits within the acquisitionprocess.Thisasynchronicitycontributestothecomplexityofmissionengineering.

It is important tonote that this skillsetaroundworking inanacquisitionenvironmentmustalsobepairedwithunderstandingthemissionenvironment(describedinSection3.1.2below).Oneof the challenges inmissionengineering is thedichotomybetweentheacquisitionviewofsystemsandthemissionviewofsystemsofsystems(describedinSection2above).Itismissionengineerswithbothskillsetsthatwerereportedtobemoveeffective.


54

3.1.2AREA2:MISSIONCONCEPT

ThefirstArea,above,providesthefoundationsforamissionengineertobeabletounderstandthe systems whichmake up amission. ThisArea defines the skills required for themissionengineertounderstandthemissionitselfandhowconstituentsystemsareexpectedtosupportandenablethemission.

2.1. Operational Context: The operational context is the combination of theconditions, circumstances, and influenceswhichwill determinewhich systemswill beused. In a Defense context, this includes the use ofmilitary forces. One of themostconsistentthemesheardthroughoutthe interviewswasthecriticalityofbeingabletounderstand all the systems that support a mission not just theoretically, but also interms of how they function and the environment(s) in which they are expected tofunction. Several interviewees stated that they hired individuals with operationalexpertisesuchasNavySealsorArmySpecialForces.Theiroperationalunderstandingiscritical to successful mission engineering. These individuals often did not have abackground in mission engineering, so their organizations trained them in missionengineeringorpairedthemwithexperiencedmissionengineers.

2.2. MissionConceptofOperations:A systemconceptofoperations (ConOps) is alensthroughwhichtoviewasystem,specificallyofhowkeyuserswillinteractwiththekey systems within a mission. Amission ConOps is the view of the critical systemsrequired to complete amissionwhich highlights how these systemswill interact at ahigh-leveltoproducethedesiredmissioneffects.

2.3. Mission Scenarios/Threads: Related to the Mission ConOps, mission threadsdefinetheend-to-endexecutionofamissionandenableindividualstounderstandhowallthesystemsofsystemsworktogether.However,asopposedtothemissionConOps,which is intentionally at a high level, mission threads include multiple levels ofabstraction and are designed to enable each teamworking on a systemor systemofsystems in the mission space to understand how to integrate with and support theoverall mission. This should help engineering teams understand the critical missionconstraintsoftheirsystemsandincorporatetheseconstraintsintotheirdesigns.

2.4. DOTMLPF Space: A critical aspect ofmission engineering is understanding notonly the systems required but also any areas where non-technical changes must bemadetoenableamission.ManyparticipantscitedtheDOTMLPF(Doctrine,Operations,Training,Materiel,Logistics,Personnel,Facility)considerationsasacriticalpieceofthemission engineer’s toolkit. Specifically, when working within the constraints of theacquisition system, where mission engineers may influence but not control systems,understandingwhenamissionneedcanbemetwithnon-technicalsolutionsorwherean existing policy or practice must change in order to meet that need, is critical tosuccessfullyimplementingmissionapproaches.


55

3.1.3AREA3:SYSTEMSENGINEERINGSKILLS

AsdescribedinSection1,therearecriticaloverlapsbetweenmissionengineeringandsystemsengineering. To that end, participants described which systems engineering skills areparticularlycriticalformissionengineers,whicharereflectedinthisArea.

3.1. Systemof Systems Engineering:Missions require a complex set of systems toworktogethertoachieveataskthat,likely,manyofthesesystemswerenotdesignedtodo. This is a classic system of systems problem. Maier (1998) defines a system ofsystems as an assemblage of components which individually may be regarded assystems,andwhichpossesstwoadditionalproperties:(a)operationalindependenceofthe components, so that if a system-of-systems is disassembled into its componentsystems,thecomponentsystemsmustbeabletousefullyoperate independently,and(b)managerial independenceofthecomponentsmeaningthecomponentsystemsnotonlycanoperateindependently,theydooperateindependently.(p.267-284)Asstatedthroughout the research interviews, system of systems engineering is highlysynonymous with mission engineering. Therefore, mastery of system of systemsprinciplesiscriticalformissionengineerstobeeffective.

3.2. Analysis:Whilesynthesis–understandinghowsystemsofsystemscancombineto deliver an overarchingmission – is important and reflected above, another criticalskillformissionengineersisthatofanalysis.Analysisistheuseofdata,simulationsandtheory tounderstandhowsomethingworks,whichmayrequirebreakingproblemsorsystemsdownintosmallerparts.

3.3. Architecture: Mission engineers consistently stated in their interviews thatarchitecting was a critical skillset for mission engineers. Being able to developarchitectureat themission level isnecessary toenable thediverse setofengineeringteams to understand their role in the broader mission context. Likewise, anunderstanding of architecture is important; mission engineers must be able tounderstand the high-level architecture of the systems of systems with which theyinteractandhow thesearchitecturesmayormaynotbecompatiblewith thedesiredmissionarchitecture.

3.4. Modeling and Simulation: Today, most individual systems are sufficientlycomplicated and complex that no one person can understand the entire systemholistically.Complexity increasesexponentiallyas individualsystemscometogethertocompleteagoalasasystemofsystems.Formostmissions,multiplesystemsofsystemsmay be required to produce the desiredmission effect. Clearly, no human being canfullyunderstandandcontrol this levelof complexity.This iswhymodeling isacriticalsupportingdisciplineformissionengineering.Modelsallowappropriatesimplificationsto bemade so that a human being can grasp the big picture of a mission. Rigorousmodelscanalsoenablemissionengineers tomake trade-offsbetween thesystemsofmultipleprograms.Effectivemodelscanalsobeusedascommunicationtoolstoenableaclearmissionvisionacrossthevarioussystemsandsystemsofsystemsinvolved.

3.5. Requirements: Finally, requirements engineering is a critical skill for mission


56

engineers.Oneofthebiggestchallengesmissionengineerscitedintheirinterviewswasthefactthatcurrentmission-levelrequirementsareseldomgeneratedand,whentheyare,theyareoftengeneratedaftermanyofthecriticalsystemsandsystemsofsystemsare already under development. The ability to clearly identify the most criticalrequirementsforamissionandcoordinatewiththesupportingsystemsandsystemsofsystems to help them understand how to meet these mission level requirements iscriticalformissionengineeringsuccess.

3.6. Integration: IEEE12207definesintegrationas“aprocessthatcombinessystemelementstoformcompleteorpartialsystemconfigurationsinordertocreateaproductspecified in the system requirements.” (ISO/IEEE 2008) In the context of missionengineering the concept expands to include combining not just system elements butentire systems or systems of systems with operational context and processes andproceedures tohelpbringamission together.Becausemissions tend to includemanydisparate systemsorSoS’s, theability tounderstandhow thepieces canandmust fittogethertoenableamissioniscritical.

3.7. GapAnalysis:Gapanalysisistraditionallythecomparisonofactualperformancewithpotentialordesiredperformance.Thisdefinitionappliesinamissioncontext,butgap analysis for mission engineering can also include the comparison of plannedperformanceforan individualsystemversustheplannedperformanceforthatsysteminamissioncontext.Akeyexampleof this inaSoS is thatof the JointTacticalRadioSystem (JTRS) and the Army’s Future Combat System (FCS). JTRS was intended toprovide the critical communications infrastructure for FCS. There were manycomplications in the integration between JTRS and FCS. However, as noted in a CFSreport,“Theinabilitytomeet…fundamentaldesignandperformancestandardsraisedconcerns that [JTRS]maynot be able to accommodate,” someof the critical uses forFCS.(CRS2005)AcomparisonofwhatJTRSwasrequiredtoprovide–requirementsthatwere set in themid-1990s – andwhere its capabilitieswere actually evolving, was acriticalactivityforFCSengineers.

There are many other systems engineering KSABCs that could support mission engineering.However,forthisreport,thecriticalskillsthatwereconsistentlycitedbyintervieweesaretheoneshighlightedhere.Asdescribed inSection3.1,thecompetencyframework isexpectedtobe tailored and additional systems engineering skills could certainly be added as part of thistailoring.

3.1.4AREA4:SYSTEMSMINDSET

SystemsMindset is primarily focused on patterns of thinking, perceiving, and approaching atask thatareparticularly relevant tomissionengineers, includingholismand integration.Thecategoriesincludedinthisareaare:

4.1. Big-PictureThinking:Alsoreferredtoas‘systemsthinking’and‘holisticthinking’,this includestheabilitytostepbackandtakeabroaderviewoftheproblemathand;


57

thisisanimportantandessentialcharacteristicofmissionengineers.‘Big-picture’couldrefertoabroaderperspectivealongmanydifferentdimensions:thesystemasawholeincluding interfacesand integration,andnot limitedtoanysub-systemorcomponent;thesystemwhileinoperation,anditsinteractionswithothersystemsandtheoperatingenvironment;theentirelifecycleofthesystem,andnotlimitedtothecurrentstageofthe system; the development program in the context of the organization and all itsother development programs; the end goal or solution to the problem at hand; theperspectives of different stakeholders; and the technical as well as the human andbusiness perspectives. Amission engineer is usually the person to bring this broaderperspective, while classic engineers and subject matter experts often tend to benarrowly focused on their area of interest. Mission engineers are not only called toprovide this big-picture perspective themselves, but also to enable others to see thisbiggerpicture.

Inadditiontothebroadcategoryof“bigpicture”thinking,therearespecifictechniquesandmindsetsthatarecriticalforthinkingholisticallyaboutsystemsofsystems.KeatingandGheorghe(2016)statethatforsystemsofsystemsthinking,thefocusisonsystembehaviorsandspecificallyonthesynergiescreatedby interactionsofspecificsystems,ratherthanfromthespecificsystemsthemselves.Becausemissionsrequiresystemsofsystemsworking together, this focuson synergistic behavior is crucially important formission engineers. This is not out of scope for general “systems thinking” but isimportanttohighlightinamissioncontext.

4.2. Adaptability: The overall ability to deal with ambiguity and uncertainty, thisinvolves the abilities to be open-minded, understand multiple disciplines, deal withchallenges, and the ability to take rational risks. By definition, experts possesscompetency inaspecificarea,which is their ‘comfortzone’;andthey typicallydonotprefer going outside that circle or comfort zone. Such experts provide value to theorganization by contributing their expertise in those focused areas.However,missionengineerstendtoshowanabilitytobroadentheircomfortzones,andgobeyondtheircurrentboundariesandtheyarealsocomfortabledoingthis.

4.3. ParadoxicalMindset:Theabilitytoholdandbalanceseeminglyopposedviews,and being able to move from one perspective to another appropriately. Typically, anengineermayhold one viewor the other, but rarelyboth. By having this paradoxicalmindset,amissionengineercontributesvaluethatisnotusuallyexpectedfromothers.Theopposing-conceptpairsare:

4.2.1. Big-PictureThinkingandAttention toDetail:Big-picture thinkingprovides thebroader higher-level perspective; at the same time, amission engineer is alsorequiredtopayattentiontothedetailsofhowthingsworkandhowtheycometogetherinasystem.

4.2.2. StrategicandOperational:Missionengineersneed tobe strategic, focusedontheendresultof‘vision’forthemission,butalsoneedtohandlethetacticalday-to-dayactivitiesanddecisionsrequiredtoreachthatvision.Theymustalsobeable to appreciate “how what is done today is going to affect things


58

downstream”.A relatedconceptpair is theability toenvision long-term issuesbut at the same time, have the drive for closurewith the current situation inordertomoveon.

4.2.3. Analytic and Synthetic: A big-picture perspective may be associated with theability tobesynthetic,andtobeabletobringtogetherand integratedifferentpiecesofapuzzle.However,amissionengineeralsoneedstobeanalyticandtobeable tobreakdownthebigpicture intosmallerpiecesonwhichotherscanfocus andwork. Todo this effectively, amissionengineerneeds tobe able tooperate at multiple levels (e.g., system, system-of-systems, and mission) andmultiple dimensions (e.g., various technical disciplines and stakeholderperspectives).

4.4. Multi-ScaleAbstraction:Theabilitytofilteroutandunderstandthecriticalbitsofinformationattherightlevelandtomakerelevantinferences.Evenwiththatfilteredinformation,missionengineersusingtheirmissionengineeringskillsneedtoknowwhentouseornotusepiecesofinformation.Suchabstractionalsoenablesmissionengineerstoconnectandextractmeaningfromdifferentstreamsofinformation;forexample,totie together information that subject matter experts of two different disciplines areproviding.

4.5. Critical Thinking: Critical thinking is the intellectually disciplined process ofactively and skillfully conceptualizing, applying, analyzing, synthesizing, and/orevaluating information gathered from, or generated by, observation, experience,reflection,reasoning,orcommunication,asaguidetobeliefandaction.(PaulandElder2008)

3.1.5AREA5:INTERPERSONALSKILLS

The fifth competency area is Interpersonal Skills. Mission engineers can not work bythemselves;theymustinteractwithavarietyofteamstoaffectchangeatthemissionlevel.Amissionengineer isexpected tobeproficient inanumberof interpersonal skills.Thespecificcategoriescontainedwithinthiscompetencyareaarelistedbelow:

5.1. Communication: Communication is critical for mission engineers since theyinteractwithavarietyofpeople,andthisisabroadcategorycoveringawidevarietyofrelated skills and abilities. Often they are an important link between individuals andgroups, both internal and external to the organization – most importantly, thecustomers andend-usersof the systembeingdeveloped.Missionengineersneed theabilitytoclearlyexpresstheirthoughtsandperspectivestoestablishasharedcommonunderstanding.

5.1.1. Audience:Missionengineersneedtocommunicatewithavarietyofdirectandindirectaudiences:customers; subjectmatterexperts;programmanagers;vicepresidents; directors; specialty engineers; problem owners; technical teams;


59

contractors;decisionmakers;systemtesters;andothersworkingonorwiththeproject.

5.1.2. Content: The variety of content thatmission engineers need to communicatecan be broadly divided into three types, based on the audience they arecommunicatingwith:

1. Technical:Communicationswithdisciplinaryandspecialtyengineersandsubject matter experts involve high technical content. Butcommunications of technical issues to managers, end-users, and otherswhomaynotbeinterestedinorwhomaybeconfusedbyallthetechnicaldetail,involvesadequateabstractionofthetechnicalcontent.

2. Managerial:Missionengineersoftenprovideproject status tomanagersand supervisors and cost-schedule constraints and expectations totechnicalpersonnel.

3. Social: Mission engineers need to maintain an amicable environmentwithin a team and to interactwith others in a courteousmanner. Suchinteractions involve communications that are neither technical normanagerialinnature.

5.1.3. Mode: Communicating the intended content to the target audience is donethroughanumberofdifferentmodes:

1. Oral:Thistakesvariousforms,dependingontheaudienceandcontext.Itcouldbeone-on-one,oraspartofateam,inperson,orremotely.

2. Presentation:Aspecial formofcommunications is theabilitytostand infrontofanaudienceandtodeliverapresentationusingappropriateaids.Further,duringpresentations,missionengineerstendtorepresentotherswho may not be in the room: they present customer needs andrequirements to others in the absence of customers, and they presentdesigndecisionsandsystemrelatedissuestocustomersintheabsenceofdesigners.

3. Writing and Documentation: Written communication skills are equallycritical for mission engineers; the scale, audience, and objective of thewritten artifact also matter. It could range from a short email tocommunicate status, to a detailed test plan, to internal documentationsupporting a project decision, to design documents being submitted forreview.

5.2. Translation: Building on the skills described under Communication, missionengineersserveacriticalroleastranslators.Theymusthelpengineersunderstandtheoperational and mission context, operators understand engineering constraints,leadership at all levels understand the constraints of these environments, and helpensure that all stakeholders understand the impacts of the acquisition systemsconstraintsontheartofwhat ispossible foragivenmission.Manymissionengineers


60

statedthatthiswasoneofthemostcriticalbenefitsmissionengineersprovide.

5.3. EnterpriseContext:Theenterprisecontextisimportantinanysystemeffort.Asmissionengineerstrytoinfluencemultipleprogramsandprojectstoalignwithrelevantmissions,theymustalsohaveanunderstandingofthepowerstructuresandprocesses–‘howworkgetsdone’– ineachof theassociatedorganizations.Without this skill set,individualswillstruggletobringaboutcriticalchangesorgarnersupportfromdecisionsmakers to enable mission development. These skills are critical to enabling missionengineerstoinfluencestakeholdersthroughoutthemissionspace.

5.4. Building&UtilizingaSubjectMatterExpert(SME)Network:Amissionengineerneedstobea‘peopleperson’,andbuildasocialnetworkofprofessionalacquaintances.Suchanetworkbecomesavaluableresourceformissionengineerstotapinto,becausethey are not expected to know answers to all problems, but rather be able to findsomeonewhohastheexpertiseandabilitytosolvetheproblem.

5.5. Coordination: In this context, coordination is the organization of the differentelements of a complex body or activity so as to enable them to work togethereffectively.Amissionengineermustbringtogetherandbringtoagreementabroadsetofindividualsorgroupswhohelptoresolvemissionrelatedissues.Thisisanenablertothe Guiding Diverse Stakehodlers competency in the next area. (Modified from thedefinitionof“coordinator”,Sheard1996andHutchisonetal.2018.)

5.6. Influence,Persuasion,andNegotiation: It iscritical foreverymissionengineerto have the skills needed tomake a point and to successfully obtain buy-in. Inmanysituations, mission engineers contribute a perspective that is different from that ofothers: a focuson theoverallmission anddirectlyon the strategicDefenseneeds. Insuch situations, it requires influence, persuasion, and negotiating skills for missionengineers toenableothers to see thebiggerpictureonwhich theyneed to focus.AsdescribedinSection1,missionengineersareoftennotempoweredwithanyauthoritytoenactthechangesrequiredtomoveasystemalreadyindevelopmenttobealignedwithamissionneed.Theymustthereforebepersuasiveandtrytoinfluenceprogramsoverwhichtheyhavenodirectcontrolorauthority.

Conflictsareboundtorise inavarietyofscenarios–acrossteams,organizations,andservices–betweenthetechnicalsideandbusinesssideoftheorganization;aswellasoutside of the organization. The mission engineer must resolve these conflicts whilekeepingthesystemgoalsinmind.Insomecases,conflictsariseduetotheexistenceofbarriers, which may be related to the organizational culture, processes, teampersonalities,orothersituationsthatcouldpreventanindividualorteamfromgettingtheirworkdone.Themissionengineerneedstheabilitytobreakthesebarriers.

3.1.6AREA6:TECHNICALLEADERSHIP

The sixth and final competency area is Technical Leadership. It is common and natural formissionengineers toplay leadership rolesatmany levelswithinanorganization.ThespecificcategoriescontainedwithinTechnicalLeadershiparelistedbelow.


61

6.1. GuidingDiverseStakeholders:Thisincludestheabilitytomanagealltheinternalandexternalstakeholders,andtokeepallteamsengagedinthemissionfocusedonthevarietyofstakeholderneeds,especiallythoseoftheenduserorcustomer.Themissionengineerisuniquelypositionedtointeractwithmanystakeholdersofthesystem–bothexternal and internal to theorganization.Being this ‘touchpoint’ person, themissionengineer needs to deal with multiple personalities, behaviors, organizations, andcultures. A key activity in this area is helping tomanage expectations on themissionneedsversustheindividualsystemneeds.

6.2. Team Building: The ability to identify, build, and effectively guide or coach ateam comprising individuals with diverse expertise, perspectives, and personalities. Amissionengineerischargedwithcoordinatingacrossprogramsandprojectstodeliveramission engineering capability. Themission engineer needs to fully know each of theteam members: their strengths, weaknesses, capacities, capabilities, limitations,personalities, expertise, and working styles. The mission engineer plays the roles ofcoach, guide, and teacher to develop the team’s capabilities and to orchestrate it toperform the required tasks. Individual leadership styles could vary, but the overallobjectiveof istoempowertheteam,toinstillconfidence,andtohelpthemtodeliverthesolutionandtobesuccessful.Anotherkeyaspectofhandlingateamistheabilitytodelegate– the leaderneeds tobuildenough trust in the team tobeable todelegatewithconfidence.

6.3. Political Savvy: Political awareness is the “ability to understand differentpeople’s agendas, and use this knowledge” to enable progress and influencingmoreeffectively and with more sensitivity to different viewpoints. (Expert ProgramManagement, 2018) Because mission engineers work with teams that span multipleorganizations – across services, domains, and a combination of government andindustry – understanding the political landscape is an important skill. This is a criticalpiece of context for the ability to Guide Diverse Stakeholders and provides criticalunderstandingrequiredforInfluence,Persuasion,andNegotiation.

6.4. Decision Making: Specifically, the skillset that enables individuals to makedecisions,especiallywithagroupofpeopleandwhen limited information isavailable.Thoughdecisionmakingrequiresinterpersonalskills,inamissionengineeringcontext,itisalsocriticalforbuildingconsensusbuildingbetweenmultipolestakeholding,requiringleadership and influence skills. For example, mission engineers may need to helpsystems engineers and program managers accept the trade-offs for their systems infavor of the mission; i.e. perhaps suboptimizing an individual system to enable thesystemtosupportthemission.

6.5. WorkforceDevelopment:Becausemissionengineeringisanemergingdiscipline,oneofthecompetenciesthatmissionengineersarecurrentlyconcernedwithishowtobuildteamsanddevelopindividualssothatthyecanperforminmissionengineering.Asthe disciplinematures, individualmission engineersmay be less concernedwith this,butatthepresenttime,thisisanimportantaspectofhowmissionengineersfunction.


62

3.2TAILORINGTHECOMPETENCYFRAMEWORK

TheMECompetencyFrameworkprovidesthebasisforcreatingatailoredversionthatcanalignwitha specificmission. Individualsmay tailor the framework specificallybasedonwhat theyhavedone – but should bemindful that all of the areas theyhavenot touched are possibleareasforfutureexploration.Organizations, likewise,couldtailortheframeworkbeforeaskingtheworkforcetoperformtheirassessments,sothatonlyareasthataredeemedcriticaltotheorganizationarecaptured.Forexample, relevantdisciplinesanddomainswillvarydependingonmissionandthespecificmissionsrequired.

Table9providestwoexamplesofhowthecompetencyframeworkcouldbetailored,basedontwo specific missions that are described in Section 1: the DoD’s Intelligence, Surveillance,Reconnaissance(ISR)missionandtheNASAJourneytoMars.Notethatwhere<notailoring>islisted, this indicates that theteamexpects thateitheranorganizationwillbeable tousethecompetencyframeworkexactlyasdefined,withnotailoringrequired,orthat forpurposesofthisexample,nospecifictailoringhasbeenidentified.

Table9.TailoringtheMissionEngineeringCompetencyModel

Area Category Mission1:ISR Mission2:JourneytoMars

1. Discipline&DomainFoundations

1.1. PrincipleandRelevantDisciplines

EE,CS/CE,ME,IT EE,ME,AeronauticalEng,Physiology,Thermodynamics

1.2. RelevantDomainsTelecom,IT, Space,logisticsandsupply

chain

1.3. SystemCharacteristicsDiviningintentionsofthinkingopponents,i.e.,highcomplexity

PioneeringapproachtoenableasustainedexpansionofhumansonMarswithuncertaintiesinfundingandemergenceofnewtechnologiesandscientificknowledge(NASA2015)

1.4. RelevantSystemsNationalGeospatial-IntelligenceAgency(NGA)systems;NavyDistributedCommonGround/SurfaceSystem(DCGS);MarineCorpsDCGS;ArmyDCGS;AirForceDCGS,SpecialOperationsForcesDCGS,IntelligenceCommunity(IC)DCGS

SLS,Orion,InternationalSpaceStation,CommercialOrbitalTransportationServices,CommercialCrewTransportationCapability

1.5. RelevantTechnologies<notailoringhighlighted> SpaceLaunchSystem(SLS)

andOrioncrewedspacecraft

1.6. AcquisitionContextCommercialtechnologyrefreshcyclevicelegacyacquisitionapproach.i.e.,faster!

Commercialservicessuchascrewandcargo

2. MissionConcept

2.1. OperationalContext<notailoringhighlighted> Expandingontherobotic

legacyforhumanexplorationonMarsthroughEarthReliantexplorationonboardtheISStodevelopdeepspace


63


systems,lifesupport,andhumanhealthreseachtoensurethesafetyandprotectionofthehumanexplorers

2.2. MissionConceptofOperations

FamiliaritywithCONOPSforISRacrossallservices.jointcommands,andintelligencecommunity

FamiliaritywithCONOPSformannedjourneytoMars

2.3. MissionScenarios/Threads

Jointintelligencepreparationoftheoperationalenvironment;Indicationsandwarning;ISRmissionmanagement;Intelligencesupporttotargeting;Battledamageassessment

AdvancetheEarthRelianthumanspaceflightprogramthroughtheProvingGroundofcislunarspacetoanEarthIndependent,deep-spacecapability(NASA2015)

2.4. DOTMLPFSpacePerDefenseIntelligenceInformatonEnterprise(DI2E)

<notailoring>

3. SystemsEngineeringSkills

3.1. SystemofSystemsEngineering

Embracingandmanaginguncertainty

<notailoring>

3.2. Analysis

3.3. Architecture

3.4. ModelingandSimulation

3.5. RequirementsEngineering

3.6. Integration

3.7. GapAnalysis

4. SystemsMindset

4.1. Big-PictureThinkinge.g.,USNavyCommanderJoeRochefortandhiscommunicationsintelligence(COMINT)teaminHawaiidiviningJapaneseNavyintentionsleadinguptotheBattleofMidwayPioneeringspaceforsustainedhumanexplorationandexpansiononMars

PioneeringspaceforsustainedhumanexplorationandexpansiononMars

4.2. AdaptabilityStandardizeforflexibilityandsimpleinterfacestoenhancecomplexsubsystems

4.3. ParadoxicalMindset<notailoring>

4.4. Multi-ScaleAbstraction

4.5. CriticalThinking

5. InterpersonalSkills

5.1. Communicatione.g,,CommanderJoeRochefortandhisCOMINTteambeforetheBattleofMIdway

EngagingallfourNASAMissionDirectoratesandallNASACenterandLaboratories,aswellasfosteringnewandexistinginternationalpublicand

5.2. Translation

5.3. EnterpriseContext


64


5.4. BuildingaandUtilizingaSMENetwork

privatepartnerships

5.5. Coordination

5.6. Influence,Persuasion,andNegotiation

6. TechnicalLeadership

6.1. GuidingDiverseStakeholder

6.2. TeamBuilding

6.3. PoliticalSavvy

6.4. DecisionMaking

6.5. WorkforceDevelopment

Table9isonlyabasicexample,butdemonstratesthattailoringcanincludetheidentificationofspecific competencies that are of critical interest to specific missions – particularly inCompetencyAreas1and2,whichareexpectedtobeheavilytailored.

3.3COMPETENCYASSESSMENTS

Inidentifyinghowcompetenciesmightbeassessed,theRT-171teamexaminedguidancefromanumberofsources,suchastheAtlasmodelcreatedbytheHelixresearchproject(Hutchisonet al. 2018) and the draft INCOSE competency model (Gelosh et al. 2017). The INCOSEcompetencymodelattemptstocreateadetaileddescriptionofeach‘proficiencylevel’(levelofcompetency attainment) for each individual competency. The Helix model takes a differentapproach.Asexplainedin(Hutchisonetal.2016):

OneoftheareasthathasprovenmoredifficultthanexpectedfortheHelixteamisthedevelopmentofarubrictoguideassessmentofproficiencies.Theteamhashelped over 100 individuals conduct self-assessments and had exploratoryconversation around these assessments, but the primary roadblock to this hasbeenthatindividualsstruggletoexplainskillsversushowtheyattainedthem.Forexample, if an individual said that they were a 6 out of 10 for “SystemsEngineering Discipline”, the team would ask what that “6” really meant. Theanswers would often be something like this: Well, I’ve been doing systemsengineeringfor5yearsandI’veseenmostofthelifecycleandIamgoodwiththetoolsweutilizehere.”Notethat“I’veseenmostofthe lifecycle”–anaspectoftheir career path – is different from “I am able to provide clear value and


65

leadership at any stage of the lifecycle.” When the team probed further,individuals simply did not have the vocabulary to describe precisely thedifferencesbetweena“5outof10”anda“7outof10”.

Intheirworktobepublishedin2018,Pyster,Hutchison,andHenrytackledthis inadifferentway. They identified a comparable competency scale which is somewhat generic – utilizingbroaddescriptionsforalevelofcompetency–ratherthantryingtotailoraspecificdefinitionfor every single topic. This is adapted from a rubric developed by the National Institutes ofHealth (NIH), the “NIH Competency Scale is an instrument used tomeasure one’s ability todemonstrate a competencyon the job. The scale captures awide rangeof ability levels andorganizesthemintofivesteps;from‘FundamentalAwareness’to“Expert’.”Pysteretal.haveadaptedthistoapplytotheAtlas framework,translatingthe levels intoa5-pointscale(1forFundamentalAwareness,2 forNovice,etc.). (2018, inprint)This is illustrated inTable 10andthe RT-171 team recommends that this is a useful approach for beginning competencyassessmentformissionengineers.

Table10.CompetencyLevels(adaptedfromPysteretal.2018,inprint,usedwithpermission)# Level LevelDescription

1FUNDAMENTAL

AWARENESSIndividualhascommonknowledgeoranunderstandingofbasictechniquesandconcepts.Focusisonlearningratherthandoing.

2 NOVICE

Individualhasthelevelofexperiencegainedinaclassroomorasatraineeon-the-job.Individualcandiscussterminology,concepts,principles,andissuesrelatedtothiscompetency,andusethefullrangeofreferenceandresourcematerialsinthiscompetency.Individualroutinelyneedhelpperformingtasksthatrelyonthiscompetency.

3 INTERMEDIATE

Individualcansuccessfullycompletetasksrelyingonthiscompetency.Helpfromanexpertmayberequiredfromtimetotime,butthetaskisusuallyperformedindependently.Theindividualhasappliedthiscompetencytosituationsoccasionallywhileneedingminimalguidancetoperformitsuccessfully.Individualunderstandsandcandiscusstheapplicationandimplicationsofchangesintasksrelyingonthecompetency.

4 ADVANCED

Individualcanperformtheactionsassociatedwiththiscompetencywithoutassistance.Theindividualhasconsistentlyprovidedpracticalandrelevantideasandperspectivesonwaystoimprovethecompetencyanditsapplicationandcancoachothersonthiscompetencybytranslatingcomplexnuancesrelatedtoitintoeasytounderstandterms.Individualparticipatesinseniorleveldiscussionsregardingthiscompetencyandassistsinthedevelopmentofreferenceandresourcematerialsinthiscompetency.


66

# Level LevelDescription

5 EXPERT

Individualisknownasanexpertinthiscompetencyandprovidesguidanceandtroubleshootingandanswersquestionsrelatedtothiscompetencyandtheroleswherethecompetencyisused.Focusisstrategic.Individualhavedemonstratedconsistentexcellenceinapplyingthiscompetencyacrossmultipleprojectsand/ororganizations.Individualcanexplainthiscompetencytoothersinacommandingfashion,bothinsideandoutsidetheirorganization.

Missionengineerscanusethisguidance inTable10tocompletetheirownassessments. InarelatedSERCproject–RT-173:Helix–thesetypesofself-assessmentshavebeenconductedforsystems engineers. TheHelix team recommends that proficiencies could be reviewed at twopointsintime:(1)presentday,and(2)apointinthepast,oftenatthestartofone’scareer.Thisenablesacompetencyprofilestobeplotted,asillustratedinFigure15.(Hutchisonetal.2018)

Figure15.ExampleCompetencyProfileforanIndividual

The competency profile is not meant to be exact since self-evaluations are subjective, andindividuals may have over- or under-rated themselves. However, this exercise enables adiscussion around the relative strengths in specific competencies; how competency levelschangedovertime;andwhatfactorsorforcescausedorenabledthosechanges.

Thisframeworkcanalsobeusedtosupportthedevelopmentoffuturemissionengineerswhowill be effective. From a competency perspective, it would mean setting target levels forcompetencyareas,asillustratedinFigure16.

0

1

2

3

4

5

Discipline&DomainFoundations

MissionConcept


SystemsMindset

InterpersonalSkills

TechnicalLeadership

StartofCareer Current


67

Figure16.ExampleCompetencyProfilewithTargetLevels

3.4MISSIONENGINEERINGTEAMS

Thecompetencyframeworkincludesallofthecompetenciesthatarerequiredtomakemissionengineeringwork.However,itisimportanttonotethatmissionengineersoftenworkinteams–sometimeswithothermissionengineersbutalwayswithgroupsofprojectmanagers,systemsengineers,disciplineengineers,etc.Therefore, itwouldbeunwiseandunrealistictotreatthecompetency as something that each individual mission engineer must attain to the highestlevel.

In the related Helix research, employees found organizational expectations for “superhero”profiles. Their rationale was that if the organization truly expected them to be expert ineverything, then there was no way they would be able to fulfill those expectations. Theseunrealisticexpectationscanmakesystemsengineers–ormissionengineers–believethattheircontributionscouldnotbevaluedbytheirorganization.(Hutchisonetal.2018b)

In some organizations from the Helix research, discussions about expectations led to therealizationthat the“minimum”waswhatwasneededfromateamof individuals,notteams.Thinkingholisticallyaboutthecapabilitiesrequiredfromateamhelpstoalleviatetheproblemofexpectingindividualstohavethehighestproficiencyineverycompetency(whichis,asstatedabove,nearlyimpossible).Figure17showsanexampleofthis:

0

1

2

3

4

5


MissionConcept


SystemsMindset

InterpersonalSkills

TechnicalLeadership

StartofCareer Current TargetLevel


68

Figure17.Exampleofteamprofilescomparedtoan“expected”profile.

Inthefigureabove,itlooksasiftheteamwouldfallshortofexpectationsinhalfofthecriticalproficiencyareas.However,teampersonalitiesanddynamicswillalsoinfluencethispicture.Itis possible that if the teamworkswell together, theywill create synergies that enable themcollectivelytobemoreeffectivethantheycouldbeindividually.AgainfromtheHelixresearch,there were several real-world examples of teams where no single individual met allexpectations,butasateam,theteamfulfilledallexpectations.This typeofapproachforunderstandingcompetencies is important inmissionengineeringaswell. Inparticular,howmissionengineers can integrate intoand influence thevarious teamsworkingonsystemswithinthemissionspacewillbecriticalforunderstandinghowtheymightbeeffective.

3.5COMPARISONTORELATEDCOMPETENCYMODELS

TheintentionhereistohighlightwherethefindingsfromRT-171areconsistentwithordifferfromexistingcompetencymodels.Ifwearenotgoingtoincludeanactualcompetencymodel,suggestwe still compare findingswith: INCOSESECompetencymodel,NASASECompetencyModel,andtheHelixSEproficiencymodel.

3.5.1MISSIONENGINEERINGCOMPETENCYANDTHEHELIX(ATLAS)SEPROFICIENCYMODEL

The formatting and approach for the Mission Engineering competency framework is basedupon the Helix (Atlas) systems engineering proficiencymodel. For this reason, it is a usefulmodeltobegincomparisons.

0

1

2

3

4

5


MissionConcept


SystemsMindset

InterpersonalSkills

TechnicalLeadership

Expected ChiefSystemsArchitect SystemsArchitect1 SystemsArchitect2


69

TheHelixmodelfocuseson6proficiencyareas(Hutchisonetal.2018):

1. Math/Science/GeneralEngineering:Foundationalconceptsfrommathematics,physicalsciences,andgeneralengineering;

2. System’sDomain&OperationalContext:Relevantdomains,disciplines,andtechnologiesforagivensystemanditsoperation;

3. SystemsEngineeringDiscipline:Foundationofsystemsscienceandsystemsengineeringknowledge;

4. SystemsMindset:Skills,behaviors,andcognitionassociatedwithbeingasystemsengineer;

5. InterpersonalSkills:Skillsandbehaviorsassociatedwiththeabilitytoworkeffectivelyinateamenvironmentandtocoordinateacrosstheproblemdomainandsolutiondomain;and

6. TechnicalLeadership:Skillsandbehaviorsassociatedwiththeabilitytoguideadiverseteamofexpertstowardaspecifictechnicalgoal.

Immediatelythereareseveralcommonareas–thoughthecategories includedintheseareasarenot identical. Systemsmindsetwas consistently citedas critical formissionengineers, aswere interpersonal skills, and technical leadership. Table 11 provides a detailed cross-walkbetweentheMEComptencyFrameworkandtheAtlassystemsengineeringcompetencymodel.

Table11.ComparisonofMissionEngineeringCompetencyFrameworkandHelix(Atlas)ProficiencyModelMEArea SEArea(Atlas) Discussion

DisciplineandDomainFoundations1.1.Principle&RelevantDisciplines

1.2.RelevantDomains

1.3.RelevantTechnologies

1.4.Complexity1.5.AcquisitionContext

Math/Science/GeneralEngineering1.1.NaturalScienceFoundations1.2.EngineeringFundamentals1.3.ProbabilityandStatistics1.4.CalculusandAnalyticalGeometry1.5.ComputingFundamentals

Forbothmodels,thesearethefoundationalskillsonwhichthedisciplineisbased.VirtuallynooneintheMEcompetencystudytalkedaboutgeneralmath,science,orengineeringskills–theywereassumedasabaseline–whereasinAtlas,systemsengineersdiscussedthisexplicitlyasthefoundationfortheirdiscipline.ForME,thefoundationswereconsideredtheknowledgeofthecriticaldomains,disciplines,andtechnologiesthatsupportthosedisciplines.SimilarlytoAtlas,theconsensusforMEwasnotthatamissionengineermustbeanexpertineachofthese,butinsteadthatamissionengineermusthave“enoughdepth”inthesetoknowledgablyengagewithSMEs.

MissionConcept2.1.MissionConceptofOperations2.2.MissionScenarios/Threads2.3.RelevantSystemsintheMissionSpace

SystemsDomainandOperationalContext2.1.PrincipalandRelevantSystems

2.2.FamiliaritywithPrincipalSystem’sConceptofOperations(ConOps)

Criticaltomissionengineeringisacleargrasponthemissionitself–whatisthegoal,howwillitbeachieved,andthecriticalsystem(s)relevanttothemission,lookingnotonlyattechnologybutotherfactorsthatwillimpactmissionsuccess,suchaslogistics,personnel,andprocesses.ThisisanalogoustotheAtlasSystemsDomainandOperationalContext.Really,thesearedifferentlensesthroughwhichtoviewasystem:forAtlasitisonthe


70

MEArea SEArea(Atlas) Discussion2.4.DOTMLPFSpace

2.3.RelevantDomains2.4.RelevantTechnologies

2.5.RelevantDisciplinesandSpecialties

2.6.SystemCharacteristics

specificsystembeingdevelopedincontextwhileformissionengineeringit’sthecontextforthesystemofsystemsthatcandeliveracapability.

SystemsEngineeringSkills3.1.SystemofSystemsEngineering3.2.Architecture3.3.Modeling3.4.Analysis3.5.RequirementsEngineering

SystemsEngineeringDiscipline3.1.Lifecycle3.2.SystemsEngineeringManagement3.3.SEMethods,Processes,andTools3.4.SystemsEngineeringTrends

ForMEandAtlas,thisareatalksabouttheimportanceofthedisciplineofsystemsengineering..Asmanydefinedmissionengineeringasstronglyrelatedtosystemsengineering(seeSection2),itisunsurprisingthatthereisclearoverlap.InAtlas,thisisawell-roundedviewacrosstheentirediscipline.Formissionengineering,however,therewerespecificfacetsofMEthatwererepeatedlyhighlighted:architecture,requirements,modelingandsimulation,analysis,andagain,thebroaderlensofsystemofsystemsengineering.Allofthesearelower-leveltopicsintheAtlasmodelbutbasedontheinterviewdata,risetotheimportanceofspecificcriticalcategoriesforME.

SystemsMindset4.1.Big-PictureThinking4.2.ParadoxicalMindset4.3.SystemofSystemsThinking4.4.Adaptability4.5.Multi-ScaleAbstraction

SystemsMindset4.1.Big-PictureThinking4.2.ParadoxicalMindset4.3.Adaptability4.4.Abstraction4.5.ForesightandVision

Here,thereisastrongcorrelationbetweentheMEframeworkandAtlasmodel.Asmanydefinedmissionengineeringasstronglyrelatedtosystemsengineering(seeSection2),itisunsurprisingthatthethoughtpatternsandapproachesthatsupportsystemsengineeringwerealsoviewedascrucialformissionengineering.Themajordifferenceherewas,again,howcriticaltheinterviewee’sbelievedtheSoSperspectiveisformissionengineers.ThisisnottosaythatbigpicturethinkingcannotincludeaSoSperspectiveforsystemsengineers–justthatformissionengineersitwasseenasconsistentlycritical.Oneoftheotherdifferencesisthat“foresightandvision”werenotdescribedinthesamewayforMEasforAtlas.Specifically,missionengineersoftendescribeddealingwithemergenceandunpredictabilitysuccessfully–theadaptability–overtheabilitytopredicthowthingswouldworkinthesecomplexSoS’s.

InterpersonalSkills5.1.Communication5.2.Translation5.3.Influence,Persuasion,and

InterpersonalSkills5.1.Communication5.2.ListeningandComprehension5.3.WorkinginaTeam

Bothmissionandsystemsengineersrequirestronginterpersonalskillstobesuccessful.Themostcommonly-citedforbothMEandAtlaswas“communication”–thoughinAtlaslisteningandcomprehensionwerecriticallyhighlightedwhereasinME,translation–theabilitytohelpmultiplestakeholdersunderstandtheviewsofotherstakeholders


71

MEArea SEArea(Atlas) DiscussionNegotiation5.4.BuildingaSocialNetwork5.5.EnterpriseContext

5.4.Influence,Persuasion,andNegotiation5.5.BuildingaSocialNetwork

by“translating”concernsacrossorganizationalanddisciplinaryboundaries–wasverycommonlycitedascritical.ForAtlas,functioninginteamwasconsistentlyhighlighted.However,inME,itisgenerallyaverysmallteamof2-3missionengineerswhoinsteadaremorefocusedoninterfacingwiththeengineeringteamsacrossanumberofprojectsandprograms.Oftentheyareoutsideoftheseprograms,andthereforeratherthanintegratingintoateammustfocusoninfluencingtheseprogramsorprojects.Systemsengineersalsoneedtoinfluence–asorganizationallytheyoftendonotcontrolotherengineers–butthisskillisrequiredatadifferentlevelformissionengineers.Bothmissionandsystemsengineersrequiretheabilitytobuildandutilizeanetworkofsubjectmatterexpertsthatcanprovidecriticaltechnicalinsights.Formissionengineers,thisnetworkalsocommonlyincludedoperatorswithin-depthknowledgeofmanyofthesystemsthatsupportamissioninpractice.Finally,whilesystemsengineersdidoccasionallyciteunderstandinghowtheirorganizationsworkasimportant,itdidnotrisetothelevelofaproficiencycategory.Incontrast,missionengineerscitedhowcriticalitistounderstandnotonlytheirownorganizationsbutalloftheorganizationsthatinteractwiththemissionsystem,includingunderstandingmultiplecommandandcontrolchainsandevensometimesacrossservices.Havingthiscontextofthebroaderenterprisewascitedascriticalforsuccessfulmissionengineering.


72

MEArea SEArea(Atlas) DiscussionTechnicalLeadership6.1.GuidingDiverseStakeholder6.2.TeamBuilding6.3.PoliticalSavvy6.4.DecisionMaking6.5.WorkforceDevelopment

TechnicalLeadership6.1.BuildingandOrchestratingaDiverseTeam6.2.BalancedDecisionMaking&RationalRiskTaking6.3.GuidingDiverseStakeholders6.4.ConflictResolution&BarrierBreaking6.5.BusinessandProjectManagementSkills6.6.EstablishingTechnicalStrategies6.7.EnablingBroadPortfolio-LevelOutcomes

Whileguidingdiversestakeholderswasimportantforsystemsengineers,itwasthemostcommonlycitedtechnicalleadershipskillformissionengineers.Inbothapproachesteambuildingwascriticallyimportant.However,formissionengineering–whichcrossesorganizationalandsometimesenterpriseboundaries–politicalsavvywasseenascriticallyimportant.Again,thisdoesnotmeanthatitisunimportantforsystemsengineeringbutthatitwasastrongtendformissionengineeringandwasnotcommonlycitedintheHelixstudy.Becausemissionengineeringisanemergingdiscipline,workforcedevelopmentandmissionengineers’criticalroleingrowingadditionalmissionengineerswascommonlycited.WhileworkforceissueswerealsocitedinAtlas,theabilitytogrowsystemsengineersdidnotemergeasacriticalcompetency.

3.5.2MISSIONENGINEERINGCOMPETENCYCOMPAREDTOADDITIONALCOMPETENCYMODELS

BecausetheshapeoftheMEComopetencyFrameworkwasinfluencedbyAtlas,itwaspossibleto do a clear mapping between the competency areas. This is not, however, feasible whencomparingtoothercompetencymodels. Instead,theteamprovidesthecomparisoninFigure18 illustrates the overlap between the ME Competency Framework and additional relatedframeworks:theINCOSESECompetencyFramework(2018inprint),theNASASECompetencies(2017), and the Navy SE CompetencyModel (Whitcom et al. 2015). Competencies from theINCOSE model are listed in blue; competencies from the NASA model are listed in green;competenciesfromtheNavySEcompetencymodelareinnavyblue.

ComparingtheMECompetencyFrameworktoothermodelsservedseveralpurposes:

• It enabled the team to identify whether there were related competencies that wereexcludedfromtheMECompetencyFrameworkand,ifdeemedcritical,toexaminethereason for the omission. For example, did the team prompt for this response in thequestions?

• ItenabledtheteamtoidentifycompetenciesthatwereuniquetotheMECompetencyframework, which helps to highlight how mission engineering differs from relateddisciplineslikesystemsengineering.

• Overlaps helped to identify key areas where mission engineering is related to otherdisciplines.


73

Figure18.ComparisonofMECompetencyFrameworkwithOtherCompetencyModels

The most obvious insight from Figure 18 is that other frameworks only lightly address thefoundational skills of their disciplines. For the Navy Competency model, only the specificdisciplineofsoftwareengineeringishighlighted.Thedisciplinesanddomainsthatarerequiredto develop missions and provide critical background to understanding the context in whichthose missions occur, are not often included in competency frameworks. But due to theirprevalence in the dataset, they are included in theME Competency Framework. It is worthnotingthattwoofthesecompetencymodelsdoaddresstheacquisitioncontextanditsimpactonthework.

TheoverlapshighlightedinFigure18arealsocriticaltohighlightingwheremissionengineeringisrelatedtootherdisciplines,inthiscasesystemsengineering.Forexample,theoverlapswiththeMissionConceptareademonstratethatgeneralsystemsengineeringisbeginningtoexpandand incorporatemission-relatedelements.This isnew in INCOSE–theCapabilityEngineeringcompetency has been added for the soon-to-be-released competencymodel – but has beenpartoftheNASAapproachtosystemsengineeringforlonger.ThisisunsurprisingasNASAhastraditionallygrownwithamissionfocusforitsengineeringefforts.

0

1

2

3

4

5

Discipline&Domain

Foundations

MissionConcept


SystemsMindset

InterpersonalSkills

TechnicalLeadership

ExampleProfile1 ExampleProfile2

CapabilityEngineering

MissionNeedsStatement

SystemEnvironments/ExternalRelationships

MissionLevelAssessment

MissionandResultsFocus

TechnicalLeadership/Leadership

TeamDynamics/TeamDynamicsandManagement/LeadingHighPerformanceTeams

StakeholderExpectationandDefinition/StakeholderRequirementsDefinition/ManagingStakeholders

Facilitation

DecisionAnalysis

CoachingandMentoring/MentoringandCoaching/CoachingandMentoring

Communication/Communication/Communication

NASAInteriorandExteriorEnvironment

Negotiation/Negotiations

Organization

CriticalThinking

SystemsThinking

StrategicThinking

ProblemSolving

Analysis/LogicalDecomposition

Architecture/SystemArchitecture/ArchitectureDesign

Modeling

Requirements/TechnicalRequirementsDefinition/RequirementsManagement/StakeholderRequirementsDefinition/RequirementsAnalysis/RequirementsManagement

ProductIntegration/Integration/InterfaceManagement/InterfaceManagement

TradeStudies

SoftwareEngineering

AcquisitionStrategiesandProcurement/Acquisition

INCOSECompetencyFramework

NASASystemsEngineeringCompetencies

NAVYSECompetencyModel

*


74

ThesystemsengineeringcapabilityareahassomeclearoverlapwithSEcompetencymodelsasexpected. It is worth noting, however, that while a subset of the systems engineeringcompetenciesrosetothetopoftheMECompetencyframework,Section1explainsthatthereis a strong and clear relationship in the data between mission engineering and systemsengineering. Therefore, there are a number of competencies contained in the othermodelsthatcouldbetailoredinottheMECompetencyFrameworkasappropriate.


75

4.VIEWSONFUTUREDIRECTIONSFORMISSIONENGINEERING

As mission engineering grows and matures, some changes are expected. The RT-171 teamaskedseveralquestionsofpracticingmissionengineersabouttheirperspectivesonthefuture.These are reported below, along with the team’s interpretation of these responses incombinationwiththeresultsoftheliteraturereview(AppendixD).

4.1MISSIONENGINEERS’PERSPECTIVESONTHEFUTURE

ThelastsetofquestionsusedintheRT-171interviewsrelatedtotheexpectationsandfuturedirections in mission engineering. In this particular section, four main questions wherecommonlyexplored.Theseinclude:

1. Whatisyourvisionforidealimplementationofmissionengineering?

2. Whathastochangetomakeitareality?

3. What do you see as the key risks to your organization developing the missionengineeringworkforceitwillneedfiveyearsfromnow?

4. Whataretheobstaclesinobtainingthesecompetencies?

Figure19.Distributionofresponsesforfuturedirectioninmissionengineering.

Figure 19 illustrates the distribution of the responses to the above questions. As it can beobserved, all the participants answered the question “What is your vision for ideal

100%

81%75%

53%

0%

20%

40%

60%

80%

100%

Future Vision of ME What needs to change? Key risks to yourorganization

What are the obstacles?

Perc

enta

ge

Distribution of Discussed Topics

Understanding Mission Engineering Future Directions (N=32)


76

implementationofmissionengineering?”.Thisquestionallowstheunderstandingoftheidealstateofmissionengineering.Thesecondmostansweredquestion(81%),Whathastochangeto make it a reality?, aimed at understanding theoretical and practical gaps in theimplementationofthemissionengineering.Then,thethirdquestion(75%),Whatdoyouseeasthekeyriskstoyouorganizationtodevelopmissionengineeringworkforce?,providesinsightsonthecurrentlimitationsinthedevelopmentofamissionengineeringworkforce.Finally,thequestion “What are the obstacles in obtaining these competencies?” received 53% ofresponses.Thisquestionaimstofacilitatetheidentificationofkeyissuesthatpracticingmissionengineersbelieveareimpactingthegrowthofamissionengineeringworkforce.Onceageneraldescriptionofthemissionengineeringfuturedirectionsdatasetwaspresented,the next step is to understand each of these questions in greater detail. The methodologyfollowedisbasedonagroundedtheoryapproach,wheretopicsemergedfromtheparticipants’responses,thenrelatedclustersofanswerswerecoded.The firstquestion,What is your vision for ideal implementationofmissionengineering?,wasansweredbytheentirepopulationofparticipants.Figure20presentstheresultsofanalyzingthetranscriptsofthisquestion.

Figure20.Distributionofresponsesforvisionofmissionengineering

Fromthefrequencyoftopicspresentedthefollowingitemscanbementioned:

• Team/Collaboration. Interviewees recognize the need of having a team within theorganizationthatfocusesonmissionengineering.

• Moreeffortsareneededwhentrainingandeducatingtheworkforce.

23%19%

13% 13%10% 10%

6%

3% 3%

0%

5%

10%

15%

20%

25%

Team/Collaboration

Training

ME Role

Modeling & Sim

ulation

Understanding M

E

SE Mindset

Processes

Requirements

ME to boards/t

asks

Perc

enta

ge

Topics

What is your vision for ideal implementation of mission engineering?(N=32)


77

• Thereisaneedforamissionengineeringroleortheformalpresenceofanindividualinchargeofthemission.

• Establishacommonunderstandingofmissionengineering.• Promoteparadoxicalthinking.• Modeling&Simulationstrategiesatthemissionlevelareneeded.• Processesthatconsiderthemissionshouldbedevelopedandimplemented.• Requirementsfordefiningwhatisneededtoimplementmissionengineeringshouldbe

defined.Next, the questionWhat has to change to make this vision a reality? was explored. As areminder, this question receives 81% of responses. Figure 21 illustrates the distribution ofresponses.

Figure21.Distributionforresponsestowhathastochangetomakeitareality

FromFigure21,thefollowingpointscanbeextracted:

• Human Capital. The hiring of personnel is needed to support mission engineeringoperations. There are limited staff resources able to perform their operations at themissionlevel.

• OrganizationalCultureasitrelatestotheprocessonhowthetasksareperformedandthewayinwhichpersonnelishired.

• Effortstodevelopmissionbasedrequirementsshouldbeinitiated.• ABookofKnowledgeonmissionengineeringwouldfacilitatecommonunderstandingof

thedomain.• Trainingandeducatingtheworkforceontopicssuchasemergenceandcomplexity.

26%

17%

9% 9% 9% 9% 9%

4% 4% 4%

0%

5%

10%

15%

20%

25%

30%

Human Capital

Culture

Requirements

Definitio

n of Miss

ion Engineerin

g

Training

ME Processes

Modeling & Sim

ulation

Test & Eva

luation

Team Collaboratio

n

Leadership

Perc

enta

ge

Topic

What has to change to make it a reality?(N=26)


78

• Processesshouldbestandardizedandembeddedincurrentprocesses.• Modeling and Simulation tools at themission level are needed especially considering

theevolutionofthemission.Currentframeworksseemtobeobsolete.

The third question,What do you see as the key risks to your organization developing themission engineeringworkforce it will need five years from now? received 75% of responses.Figure22presentstheperceivedriskstodevelopamissionengineeringworkforce.

Figure22.Distributionofresponsestorisksinmissionengineering

Fromtheaboveillustration,thefollowingitemscanbeextracted:

• Human Capital. The risk of finding the personnel with the right knowledge andaptitudes.Thelevelofcomplexitymissionengineeringistacklingcallsforpeopleacrossmultipledomainswithveryspecificknowledge.Thus,multidisciplinaryteamsneedtobebuilt.

• Missionengineeringisbeingaddressedwithprocessesthatareobsoleteduetothelevelofcomplexityatthemissionlevel.

• Value of Mission Engineering. Participants mentioned that in some instancesmanagement does not recognize that value of ME, making its implementation andfundingachallenge.

• Strongandsupportiveleadershipisneeded.• Toolsatthemissionlevelareneeded.Existingmodelingandvisualizationarenotableto

copewithsuchlevelsofcomplexity.• Culture risks. Relate to individuals not being confident andmotivated tobe effective.

Willingnesstostepoutsidecomfortzone.

35%

15%

10% 10% 10% 10%

5% 5%

0%

5%

10%

15%

20%

25%

30%

35%

40%

Human Capital Processes Value ofMission

Engineering

Leadership Tools Culture Requirements UnderstandingME

Perc

enta

ge

Topic

What do you see as the key risks to your organization developing the mission engineering workforce it will need five years from now?

(N=24)


79

• Lackofunderstandingofmissionengineeringprinciples.

Lastly,Figure23illustratesthemostdiscussedfactorsthatarelimitingtheacquisitionofthesecompetencies.

Figure23.Distributionofresponsestoobstaclesinobtainingthecompetencies

FromFigure23,itcanbeobservedthat:

• Humancapital includethosetopicsthat influence,supportor impacttheperformanceof the individual. Lack of employee self-motivation was reported as an obstacle. Inaddition,furthereffortsondevelopingcompetenciesareincentivized.

• It is difficult to implement any changes since the current culture resist to changes. Ashiftinthemindsetofemployeesismentioned.

• Training, there is a need to train systems engineering across multiple disciplines.Training in various domains is needed since the complexity of mission engineeringprojectsismultidisciplinary.

• Modeling&Simulation.Thearchitectureframeworkscurrentlyusedareconsideredtobeobsolete.Participants suggest transitioning toanenterprise levelarchitecture thankeeprelyingonstaticframeworks.

• Competition with Industry. It has been recognized that industry has a competitiveadvantagewhen recruiting talent. Nowadays, fewer individuals with strong analyticalskillsareattractedtothegovernmentjobmarket,thusnewstrategiesareneededwhenaattractingnewhires.

• Currenttransitionperiodthustherearelessavailableacquisitionprograms.

33%

20%

13% 13%

7% 7% 7%

0%

5%

10%

15%

20%

25%

30%

35%

Human Capital Culture Training Modeling &Simulation

None Competing withIndustry

Availableacquisitionprograms

Perc

enta

ge

Topic

What are the obstacles in obtaining these competencies?(N=17)


80

4.2MISSIONENGINEER’SFUTUREVISIONFORMISSIONENGINEERING

ThefuturevisioninMissionEngineeringsectionwaselaboratedbasedontheresponseof100%ofparticipants.A common agreement among participants is that efforts to have a common definition ofmissionengineeringareneeded. Itemssuchabookofknowledgewouldserveasareferencepoint for an emerging community. Also, practitioner mission engineers foresee thedevelopment of mission engineering teams. There are limited to none instances ofmultidisciplinaryteamsthatfocusatthemissionlevel,thusit isforeseenthatnewteamsandpartnership emerge when addressing mission level operations. Similarly, it is expected thatcurrentworkforcereceivestrainingandincentivestounderstandmissionconcepts.Inaddition, themissionengineering rolehasnotbeingdefined, thus it isexpected that suchrole gains recognition as the discipline evolves. With respect to modeling and simulationtechniques, toolsandtechniquesdedicatedtothemission levelareexpectedtobeavailable,existingmethodsareinsufficienttocopewithsuchlevelsofuncertaintyandcomplexity.Lastly,thevisionformissionengineeringtobeembeddedintotheprocesses.Inordertoachievesuchvision,expertsrecognizethatmissionengineeringhastobediscussedinmultipleboards.

4.3MISSIONENGINEER’SPERSPECTIVESONFUTURECHALLENGES

Identified pain points in mission engineering were classified into human capital, value ofmissionengineering,culture,andtools.Acentralchallenge isthecommonunderstandingofmissionengineering.Multipledefinitionsand lackofknowledgemakethisdisciplinedifficult tobevalued.Topromoteandextendthediscipline, experts suggest presenting mission engineering in multiple boards as well as thecreationofaknowledgerepositorythatservesareferencepoint.Another common discussed challenge relies on the current workforce. Finding, training,developingandmotivatingthepersonnelisconsideredoneofthemajorareasofopportunityinmissionengineering.Thelevelofcomplexitymissionengineeringistacklingcallsforindividualsacross multiple domains with very specific knowledge, therefore personnel with the rightknowledgeandaptitudesshouldbehired.However,thedisciplineisnotwellrecognizedamongpractitionersorleadershipsothereisnomotivationforsteppingoutfromthecomfortzone.Inaddition,Itisdifficulttoimplementanychangessincethecurrentcultureresisttochanges.To complicate matters further, the competition with industry for individuals with stronganalytical skills is bringing new challenges. Fewer candidates are considering a career in


81

government related projects due to the fact that government are considered be lesscompetitive.Therefore,newstrategiesareneededwhenattractingnewhires.

Lastly, the levelof complexityofmissionengineeringmakesexistingprocessesandmodelingand simulation tools obsolete. Experts suggest that visualization methods which includeevolvingscenariosareworthinvestigating.


82

CONCLUSIONS

Mission engineering is a relatively new and evolving discipline. The findings of this studywerederived from interviews with 32 mission engineers, an extensive literature review, andexaminationofcurrentacademiccourses inmissionengineering.Thedata indicatethatmissionengineering is systems of systems engineering plus additional activities (stated by 70% ofparticipants).Theskillsnecessary foramissionengineerparallel theadvancedtechnicalskillsofsystemsofsystemsengineeringwithmoreofanemphasisonoperationaldomainknowledge.

Thenon-technicalchallengesformissionengineeringwererelatedtoissueswiththerecognitionoftheimportanceofmissionengineering’sroleinmissions,havinganorganizationalidentityformissionengineeringandacquiringthenecessaryfundingandworkingthroughgovernanceissues.Thetechnicalchallengesincludetheoperationalcontext,requirementsmanagementandtesting;notunsurprisingforasystems-orientedacquisitiondiscipline.

This study does not mark an endpoint on mission engineering but rather a waypoint. Someconclusionsfromthisstudyare:

• Missionengineeringneedstobefundedasauniqueorganizationalentityinoperationsoratleastrecognizedasaseparatediscipline.

• ThereisaneedforeducatingDoDpersonnelonwhatMissionEngineeringis,but,inordertodothat,moreisneededindevelopingtheappropriatecourseworkandmaterials.Thisisnecessaryforpersonnelinboththeacquisitionandoperationalcontexts.

• MissionengineeringasSystemsofSystemsEngineeringrequirespersonnelnotonlywithstrong technical skills in areas such as modeling and simulation but also stronginterpersonalskillswellintegratedintheoperationalorganizations.

• Acquisitionneedstodrawfromtheexpertiseofthemissionengineers.

• Moreextensivestudiesneedtofurtherexploretheprocessesofmissionengineeringandthe skills and talents necessary for the process. This would include observing missionengineering in practice and further interview studies to continue the modeling of thisdiscipline.


83

ACRONYMSANDGLOSSARY

ACRONYMS

AIA AerospaceIndustriesAssociation

AIAA AmericanInstituteofAeronauticsandAstronautics

DACAS DigitallyAidedCloseAirSupport

DARPA DefenseAdvancedResearchProjectsAgency

DCGS DistributedCommonGround/SurfaceSystem

DI2E DefenseIntelligenceInformationEnterprise

DoD DepartmentofDefense

DOTMLPF Doctrine,Organization,TraininglMateriel,Logistics,Personnel,Facilities

FAA FederalAviationAdministration

IAMD IntegratedAirMissileDefense(Army)

IC IntelligenceCommunity

IEEE InstituteofElectricalandElectronicsEngineers

INCOSE InternationalCouncilonSystemsEngineering

ISR Intelligence,Surveillance,Reconnaissance

MDA MissileDefenseAgency

ME MissionEngineering

MORS MilitaryOperationalResearchSociety

NASA NationalAeronauticsandSpaceAdministration

NDIASED NationalDefenseIndustrialAssociationSystemsEngineeringDivision

NGA NationalGeospatial-IntelligenceAgency

NLCC NationalLeadershipCommandCapability

NTSB NationalTransportationSafetyBoard

PM ProjectManagement

SE SystemsEngineering

SoS SystemofSystems


84

SoSE SystemofSystemsEngineering

UKMOD UnitedKingdomMinistryofDefence

GLOSSARY

excerpt aselectionofinterviewdatawhichaddressesaspecifictopicortheme,whichisidentifiedbycodesthathighlightthespecificcontentoftheremark.

mission thetask,togetherwiththepurpose,thatclearlyindicatestheactiontobetakenandthereasontherefore(DoD2017)

missionanalysis tounderstandaproblemoropportunity,analyzethesolutionspace,andinitiatethelifecycleofapotentialsolutionthatcouldanswertheproblemortakeadvantageoftheopportunity(SEBoK2017)

missionengineering Thedeliberateplanning,analyzing,organizing,andintegratingofcurrentandemergingoperationalandsystemcapabilitiestoachievedesiredwarfightingmissioneffects(DAG2017)

system afunctionally,physically,and/orbehaviorallyrelatedgroupofregularlyinteractingorinterdependentelements;thatgroupofelementsformingaunifiedwhole(DoD2017)

systemengineering amethodicalanddisciplinedapproachforthespecification,design,development,realization,technicalmanagement,operationsandretirementofasystem(DoD2017)

systemofsystems asetofarrangementofsystemsthatresultswhenindependentandusefulsystemsareintegratedintoalargersystemthatdeliversuniquecapabilities(DoD2017)

systemofsystemsengineering planning,analyzing,organizing,andintegratingthecapabilitiesofnewandexistingsystemsintoaSoScapabilitygreaterthanthesumofthecapabilitiesofitsconstituentparts(DoD2017)


85

APPENDIXA:RT-171PUBLICATIONSANDPRESENTATIONS

Hutchison, N., G. Vesonder, D. Verma, H. See Tao, W. Miller. 2017. “Mission EngineeringCompetency.” Presentation to the Northrup-Grumman INCOSE Community of Practice(COP),23August2018.

Miller,W.2017. “RT-171:MissionEngineeringCompetencies.”Presented to the InternationalCouncilonSystemsEngineering(INCOSE)SystemofSystemsWorkingGroupattheINCOSEInternationalSymposium,15July2017,Adelaide,Australia.

Miller,W.,Y.SeeTao,G.Vesonder,D.Verma,N.Hutchison,J.Wade.2018.“DevelopmentofaDefenseMissionEngineeringCompetencyModel.”PresentedtotheInternationalCouncilonSystems Engineering (INCOSE) System of Systems Working Group at the INCOSEInternationalWorkshop,21January2018,Jacksonville,Florida.

SeeTao,H.,G.Vesonder,D.Verma,N.Hutchison,W.Miller,J.Wade.2017.“DevelopmentofaDefenseMissionEngineeringCompetencyModel.”Presentedat theUniversityofTexas,ElPaso(UTEP).

Vesonder, G. 2017. “RT-171: Mission Engineering Competencies.” Presented to the SystemsEngineering Research Center (SERC) Executive Advisory Board (EAB) 12 July 2017 inWashington,DC.

Vesonder,G.,D.Verma,N.Hutchison,H.SeeTao,W.Miller.2017.“DevelopmentofaDefenseMissionEngineeringCompetencyModel.”Proceedingsofthe20thAnnualNationalDefenseIndustrial Association (NDIA) Systems Engineering Conference, October 23-26, 2017,Springfield,VA.

Vesonder,G.,D.Verma,N.Hutchison,H.SeeTao,W.Miller.2017.“DevelopmentofaDefenseMission Engineering CompetencyModel.” Presented at the Systems Engineering ResearchCenter (SERC)’sAnnual SERC SponsoredResearchReview, 7November2018,Washington,D.C.

AcceptedforPublicationMiller,W.,N.Hutchison,H.SeeTao,D.Verma,G.Vesonder.2018(expected).“Frameworkfor

Mission Engineering Competencies.” To be presented at the 28th Annual INCOSEInternationalSymposium,7-12July2018,Washington,D.C.

Miller,W.,G.Vesonder,D.Verma,H.SeeTao,N.Hutchison,S.Luna.2018(expected).“MissionEngineering Competencies”. To be presented at the American Society for EngineeringEducation(ASEE)AnnualConferenceandExposition,24-27July2018,SaltLakeCity,Utah.


86

APPENDIXBCITEDANDRELATEDREFERENCES

Aguilar,J.A.2009.DesignAssuranceGuide.TheAerospaceCorporation,TOR-2009(8591)-11,4June.

Azani,C.,2008,AnOpenSystemsApproachtoSystemofSystemsEngineering,SystemofSystemsEngineering-Innovationsforthe21'Century,(M.Jaroshidi,Ed.),JohnWileySeriesonSystemsEngineering,NewYork,2008

Behre,C.AppliedJointMissionThreads,USJointForcesCommand,DoDEnterpriseArchitectureConference.

Brown,R.B.2017.“TheIndo-AsiaPacificandtheMulti-DomainBattleConcept”,MilitaryReview,March.

Butler,G.andWoody,C.,2017.“MissionThreads:BridgingMissionandSystemsEngineering”,SoSECIEWebinarPresentationonJune20,2017

Carlson,E.2013.JoeRochefort’sWar,reprintedition,Annapolis,US-MD,NavalInstitutePress.

Collins,B.,S.Doskey,andJ.Moreland.2017.“ModelingtheConvergenceofCollaborativeSystemsofSystems:AQualitativeCaseStudy”,SystemsEngineering,WileyPeriodicals,Hoboken,US-NJ.

Cook,S.,S.Nowakowski,andM.Unewisse.2012.ASystem-of-SystemsEngineeringApproachforAustralianLandForceCapabilityIntegration.AustralianDepartmentofDefence,DSTO-TR-2743,September.

CRS.2005.TheJointTacticalRatiodSystem(JTRS)andtheArmy’sFutureCombatSystem(FCS):IssuesforCongress.Washington,DC:CongressionalResearchService.November17,2005.

DAG.2017.DefenseAcquisitionGuidebook.Washington,D.C.:Pentagon

Dahmann,J.S.andBaldwin,K.2008.UnderstandingTheCurrentStateOfUSDefenseSystemsOfSystemsAndTheImplicationsForSystemsEngineering.ProceedingsOfTheIEEEInternationalSystemsConference.MontrealCanada.2008

Dahmann,J.S.andBaldwin,K.2011.ImplicationsofSystemsofSystemsDesignandEngineering.ProceedingsofTheIEEEInternationalConferenceonSystemofSystemsEngineeringConference.Albuquerque,NM,2011

Dahmann,J.S.,2012.“INCOSESoSWorkingGroupPainPoints”.InProcTTCP-JSA-TP4Meeting.Retrievedfromhttp://www.ndia.org/Divisions/Divisions/SystemsEngineering/Documents/NDIA-SE-MS-SoS_2013-08-20_Dahmann.pdf

Dahmann,J.S.,2018.“SystemsofSystems(SoS)EngineeringTechnicalApproachesAsAppliedtoMissionEngineering”.SoSECIEWebinarPresentationonJanuary9,2018.Retrievedfromhttps://www.acq.osd.mil/se/webinars/2018-01-09-SoSECIE-Dahmann-brief.pdf


87

Dahmann,J.S.,et.al,2012.“IntegratingSystemsEngineeringandTestandEvaluationinSystemofSystemsDevelopment”.ProceedingsoftheIEEEInternationalSystemsConference(SysCon).Vancouver,BC.2012

Dahmann,J.S.,et.al.,2010.“SystemsEngineeringArtifactsforSoS”.ProceedingsofInstituteofElectricalandElectronicsEngineersSystemsConference,2010,SanDiego,CA.

Dahmann,J.S.,et.al.,2010.“SystemsofSystemsTestandEvaluationChallenges”.Proceedingsofthe20105thInternationalConferenceonSystemofSystemsEngineering,2010,Loughborough,U.K.

Dahmann,J.S.,et.al.,2011.“AnImplementers’ViewofSystemsEngineeringforSystemsofSystems”,2011IEEEInternationalSystemsConference,Montreal,2011,Canada.

Deiotte,R.,andR.K.Garrett,Jr.2013.ANovelApproachtoMission-LevelEngineeringofComplexSystemsofSystems;AddressingIntegrationandInteroperabilityShortfallsbyInterrogatingtheInterstitials,MissileDefenseAgency,13-MDA-7269,29April.

DepartmentofHomelandSecurity,UnitedStatesCoastGuard.2014.OperationalRequirementsDocument(ORD)forNationwideAutomaticIdentificationSystem(NAIS),Version2.0,September.

DoD.2006.Acquisition:SystemsEngineeringPlanningfortheBallisticMissileDefenseSystem(D-2006-060),2March.Washington,D.C.:Pentagon,USDepartmentofDefense.

DoD.2008.SystemsEngineeringGuideforSystemsofSystems.Washington,D.C.:Pentagon,USDepartmentofDefense.

DoD.2017.DefenseAcquisitionGuidebook(DAG).Washington,D.C.:Pentagon,USDepartmentofDefense.

DoD.2018.DoDDictionaryofMilitaryandAssociatedTerms.Washington,D.C.:Pentagon,USDepartmentofDefense.

Dombkins,D.,2007.“WavePlanning”.ComplexProjectManagement,Booksurge,Publishing,SouthCarolina

Garrett,R.K.,Jr.,S.Anderson,N.T.Baron,andJ.D.Moreland,Jr.2011.“ManagingtheInsterstitials,aSystemofSystemsFrameworkSuitedfortheBallisticMissileDefenseSystem”,SystemsEngineeringVol.14,No.1,pp.87-109,J.Wiley,Hoboken,US-NJ.

Gold,R.,2016.“MissionEngineering”,Presentationatthe19thNDIASystemsEngineeringConference,October24-27,Springfield,VA

Guarro,S.B.,G.A.Johnson-Roth,andW.F.Tosney.2012.MissionAssuranceGuide.TheAerospaceCorporation,TOR-2007(8546)-6018RevisionB,ElSegundo,US-CA,1June.

Hue,M.2014.AnAnalysisofSEandMBSEConceptstoSupportDefenceCapabilityAcquisition.AustralianDepartmentofDefence,DSTO-TR-3039,September.

Hutchison,N.,D.Verma,P.Burke,M.Clifford,R.Giffin,S.Luna,M.Pertacz.2017.Atlas1.1:TheTheory of Effective Systems Engineers Revised. Hoboken, NJ: Systems EngineeringResearchCenter(SERC),StevensInstituteofTechnology.SERC-2018-TR-101-A.


88

Hutchison,N., D. Verma, P. Burke,M. Clifford, R. Giffin, S. Luna,M. Pertacz. 2017.Atlas 1.1Implementation Guide: Moving from Theory into Practice. Hoboken, NJ: SystemsEngineering Research Center (SERC), Stevens Institute of Technology. SERC-2018-TR-101-B.

ISO/IEC/IEEE 15288: 2015 Systems and software engineering – system life cycle processes.Retrievedfromhttps://www.iso.org/standard/63711.html

ISO/IEC. 2009. Systems and Software Engineering Vocabulary (SEVocab) - ISO/IEC 24765. inInternational Organization for Standardization (ISO)/International ElectrotechnicalCommission(IEC).Geneva,Switzerland,2009.

ISO/IEEE. 2008. Systems and Software Engineering - Software Life Cycle Processes. Geneva,Switzerland: International Organization for Standards (ISO)/Institute of Electrical &ElectronicsEngineers(IEEE)ComputerSociety,ISO/IEEE12207:2008(E).

ISO/IEC 20000-1: 2011. Information technology – Service management – Part 1: Servicemanagement systemrequirements. InternationalOrganization forStandardizationandInternationalElectrotechnicalCommission.

Jamshidi,M.2009.IntroductiontoSystemsofSystems,inSystemsofSystemsEngineering–Innovationsforthe21stCentury.Wiley,Hoboken,NJ.

JohnsHopkinsAPL.2002.AirDefensePartIII–BattleForceEngineering,JohnsHopkinsAPLTechnicalDigest,Vol.23,Nos2and3,April–September.

JointChiefsofStaff.2014.CloseAirSupport,JointPublication3-09.3,25November.

Kluth,V.2016.“MissionThreadsintheDI2E”,presentationatthe19thAnnualNDIASystemsEngineeringConference,Springfield,US-VA,27October.

Lee,S.2010.DoDAFV2.0FitforPurposeExampleCapabilityAnalysis,DoDCIO,11May.

Luzeaux,D.,Ruault,J.-R.,2008.SystemsofSystems.JWiley,London

Maier,M.,1998.“ArchitectingPrinciplesforSystems-of-Systems”,SystemsEngineeringVol.1,No.4,pp.267-284

Marvin,J.W.,andR.K.Garrett,Jr.2014.“QuantitativeSoSArchitectureModeling”,paperpresentedatComplexAdaptiveSystemsConference,Philadelphia,US-PA,ProcediaComputerScience36,pp.41-48.

Marvin,J.W.,J.T.Schmitz,andR.A.Reed.2016.“Modeling-Simulation-Analysis-Looping:21stCenturyGameChanger”,paperpresentedatthe26thINCOSEInternationalSymposium,Edinburgh,Scotland,UK,18-21July.

Marvin,J.,T.Whalen,B.Morantz,R.Deiotte,andR.K.Garrett,Jr.2014.“UncertaintyQuantification(UQ)inComplexSystemofSystems(SoS)ModelingandSimulation(M&S)Environments”,paperpresentedatthe24thINCOSEInternationalSymposium,LasVegas,US-NV,30June–3July.

Mindell,D.A.2015.OurRobots,Ourselves:RoboticsandtheMythofAutonomy,Viking,NewYork,US-NY,Chapter4War.


89

MITRE.SpanningtheOperationalSpace:HowtoSelectUseCasesandMissionThreads.

Moreland,J.D.,Jr.2009.:StructuringaFlexibe,AffordableNavalForcetoMeetStrategicDemandinthe21stCentury”,NavalEngineersJournal,No.1,AmericanSocietyofNavalEngineers,pp.35-51.

Moreland,J.D.,Jr.2014.“ExperimentalResearchandFutureApproachonEvaluatingService-OrientedArchitecture(SOA)ChallengesinaHardReal-TimeCombatSystemEnvironment”,SystemsEngineering,Vol.17,No.1,pp.52-61,J.Wiley,Hoboken,US-NJ.

Moreland,J.,2015.“MissionEngineeringIntegrationandInteroperability(I&I)”,NavalSurfaceWarfareCenter,DahlgrenDivision,LeadingEdge

Moreland,J.andJ.Thompson.2017.“DepartmentofDefenseMissionEngineering60-DayPilots.”PresentedattheDoDSystemsEngineeringForum,22August2017.

Mullins,M.andC.Beattie.2009.JointCoordinatedImplementationofDigitally-AidedCASCapability,presentedatInternationalDataLinkSymposium.

NASA,2015,“NASA’sJourneytoMars:PioneeringNextStepsinSpaceExploration”.Retrievedfromhttps://www.nasa.gov/sites/default/files/atoms/files/journey-to-mars-next-steps-20151008_508.pdf

NASA,2016,“NASASystemsEngineeringHandbook”.Retrievedfromhttps://www.nasa.gov/sites/default/files/atoms/files/nasa_systems_engineering_handbook_0.pdf

NASA,2017,“MissionEngineeringandSystemsAnalysisDivision(MESA)”.Retrievedfromhttps://aetd.gsfc.nasa.gov/590/index.php

NDAA,2016,“NationalDefenseAuthorizationActforFiscalYear2017”.Retrievedfromhttps://www.congress.gov/114/crpt/hrpt840/CRPT-114hrpt840.pdf

Neaga,E.I.,Hensaw,M.J.d.,andYue,Y.,2009.“TheInfluenceoftheConceptofCapability-BasedManagementontheDevelopmentoftheSystemsEngineeringDiscipline”,Proceedingsofthe7thAnnualConferenceonSystemsEngineeringResearch,April20-23,2009,LoughboroughUniversity,Loughborough,England,UK.

ODU,2017,“GraduateCertificateinMissionAnalysisandEngineering”.Retrievedfromhttps://www.odu.edu/cepd/certificates/mission-engineering

Ondrus,P.andFatig,M.,1993.“MissionEngineering”.NASAGoddardSpaceFlightCenter,Greenbelt,MD.Retrievedfromhttps://ntrs.nasa.gov/search.jsp?R=19940019408

Ott,W.,B.A.Olson,andP.Blessner.2014.AnAnalysisofAnIntegratedSystemsEngineering&Test&EvaluationApproachfor“Collaborative”SystemofSystems,presentedatINCOSESEDC.

OUSDAT&L,2008.SystemsEngineeringGuideforSystemsofSystems.Washington,D.C.:Pentagon

Paul,R.andL.Elder.2008.TheMiniatureGuidetoCriticalThinkingConceptsandTools.FoundationforCriticalThinkingPress.


90

Powers,B.2014.NormalizingDigitalCloseAirSupport(DCAS):Needs,Challenges,andtheRoleofUnmannedSystems,UnitedStatesJointForcesCommand,presentedattheWesternUASConference,11September.

Schofield,D.M.2010.AFrameworkandMethodologyforEnhancingOperationalRequirementsDevelopment:UnitedStatesCoastGuardCutterProjectStudy,MasterofScienceinEngineeringandManagementthesis,MIT,May.

SEBoK,2017,“GuidetotheSystemsEngineeringBookofKnowledge(SEBoK)”.Retrievedfromhttp://www.sebokwiki.org/wiki/Guide_to_the_Systems_Engineering_Body_of_Knowledge_(SEBoK)

SERC,2017,“WorldwideDirectoryofSystemsEngineeringandIndustrialEngineeringPrograms”.Retrievedfromhttp://www.sercuarc.org/world-wide-directory/

Sheard,S.1996.“TwelveSystemsEngineeringRoles.”ProceedingsoftheInternationalCouncilonSystemsEngineering(INCOSE)SixthAnnualInternationalSymposium,Boston,Massachusetts,USA,1996.

Spaulding,C.R.,W.S.Gibson,S.F.Schreurs,D.L.Linsenbardt,andA.DeSimone.2011.SystemsEngineeringforComplexInformationSystemsinaFederated,RapidDevelopmentEnvironment,JohnsHopkinsAPLTechnicalDigest,Vol.29,No.4,pp.310-326.

Stafford,J.C.2017.“HowtoBuildanArmadillo:LessonsLearnedfromtheFirstForward-DeployedTHAADBattery”,MilitaryReview,March.

Snowden,D.J.andM.E.Boone.2007.“ALeader’sFrameworkforDecisionMaking.”HarvardBusinessReview.November2007.

Wang,H.D.-M.Pei,L.Gan,R.-Z.Chen,andZ.Li.2017.“AShortReviewofU.S.NavalShipConceptDesignTechnologyDevelopmentFeatures”,Našemore64(2),pp.69-76.

Whitcomb,C.A.,J.Delgado,R.Khan,J.Alexander,C.White,D.Grambow,P.Walter.2015.“TheDepartmentoftheNavySystemsEngineeringCareerCompetencyModel.”Proceedingsofthe12thAnnualAcquisitionResearchSymposium.

Whitcomb,C.A.,C.White,R.Khan,D.Grambow,J.Velez,andJ.Delgado.2017.“TheU.S.DepartmentofDefenseSystemsEngineeringCompetencyModel”,paperpresentedatthe27thINCOSEInternationalSymposium,Adelaide,AU,15-20July.

Woody,C.andC.Alberts.2014.“EvaluatingSecurityRisksUsingMissionThreads”,CrossTalk,September-October.


91

APPENDIXC:DETAILEDMETHODOLOGY

Themethodology forRT-171wasmodeledafter theRT-171researchmethodology.TheHelixproject (currently SERC RT-173) has been ongoing since 2012 and has evolved over time amixed methods methodology incorporating grounded theory and more traditional researchapproaches. Because the Helix research included exploration of the state of practice forsystemsengineeringaswellasthesuccessfuldevelopmentofacompetency(i.e.competency)model for systems engineering, thismethodologywas deemed appropriate for an analogousexplorationofmissionengineering.

Theprocessanddatasetareoutlinedinthesectionsbelow.

C.1 RT-171RESEARCHPROCESS

TheRT-171researchmethodologydiscussed intheprecedingsectionwasdeployedusingtheresearch process illustrated in Figure 24 below. The RT-171 research process consists of sixmajorsteps:

A. PreparationB. CollectionC. AnalysisD. AnswerResearchQuestionsE. PublishResultsF. MethodologyReview


92

Figure24.RT-171ResearchProcess

ThefocusofRT-171wasonexecutingtheloopA-B-C-F-Amultipletimeswithindividualsfromdifferentorganizations.TheloopB-C-Bwasexecutedafewtimeswheninterviewswereconductedwithindividualsconsideredthoughtleadersinmissionengineering.InadditiontoperformingtheA-B-C-F-Aloopwithnewindividuals,stepsD-Ewereexecutedthatledtodevelopmentofthepreliminaryfindingsinthisreport.

C.1.1 PREPARATIONFORDATACOLLECTION(A)

Preparationfordatacollectionwasthefirststepexecutedintheproject.Initially,organizationsfromwithin the US DoDwere identified for data collection; also, the primary focus was onmission engineers in these organizations (A1). As RT-171 progressed, non-DoD governmentorganizationswere added, aswere thought leaders from non-governmental organizations. Aproject overview, including intent, purpose, and research objectives was provided toparticipants(A2).

C.1.2 DATACOLLECTION(B)

Followingapprovedresearchprotocols,asignedconsentformiscollectedfromtheparticipantsbefore conducting interviews (B1). Based on their willingness to participate in RT-171interviews,individualswerescheduledattheirconvenience.Mostoftheeffortsfor2017werefocusedongatheringdatafrommissionengineeringpractitioners(B2).ThedatasetisdescribedinSectionC.2,below.Theseinterviewsfocusedonanumberofquestions,coveringanumber

D.ANSWERRESEARCHQUESTIONS

D1.CompetencyFramework

A.PREPARATION

A1.Iden&fyOrganiza&onsand

Individuals

A2.ProvideProjectMaterialstoPar&cipants

E.PUBLISHRESULTS

E1.TechReports

E2.Conference/JournalAr&cles&Presenta&ons

B.COLLECTION

B1.CollectConsentForms&Resumes

B2.ConductDataGatheringInterviews

(90min)

F.METHODOLOGYREVIEW

F1.Iden&fyUpdatesforInterviewQues&ons

F3.Iden&fyUpdatesforDataCollec&onApproach

C.ANALYSIS

C2.Ini&alAnalysis

C1.InterviewSummaries

C3.PreliminaryFindings

C4.DetailedAnalysis

B3.ConductValida&on/ReviewInterviews

(60min)

D2.AnswerResearchQuesLons


93

ofareassuchas:• Thedefinitionandscopeofmissionengineering:

o Howtheindividualcametoworkinmissionengineeringo The individual and organization’s definition and philosophy around mission

engineeringo Commonprocessesandpracticesformissionengineeringintheorganizationso Overlapsbetweenmissionengineeringandsystemsengineeringo Criticalactivitiesinmissionengineeringo Criticalchallengesinmissionengineering

• Thecriticalcompetenciesrequiredformissionengineering:o Selectionofindividualswhowillbecomemissionengineerso Criticalskillsrequiredforsystemsengineerso Qualitativevalueofcriticalskills,includinghowusefultheyhavebeenpersonally

andhowcommontheyareintheexistingmissionengineeringworkforce• Viewonfuturedirectionsformissionengineering:

o Anticipatedcriticalchallengesandcriticalriskso Includingwhatneedstochangeinordertomeetthesechallenges

Interviewswerenomorethan60minutesinlength.ThequestionsusedintheinterviewscanbefoundinsectionC.1.3,below.Finally, the team conducted interviews with individuals who were deemed to be thoughtleaders in mission engineering. During these interviews, current analysis and insights weresharedandexpertsaskedtoprovidetheirfeedbackandageneral“sanitycheck”ontheresults.InterviewQuestionsRT-171utilizedaninformalinterviewstyleinwhichquestionswereusedtopromptdiscussion,butintervieweeswerenotrigidlyrequiredtoansweronlythequestionsasasked.Notethatnotallquestionswereaskedofeachinterviewee.Forexample,someonewhoiscurrentlypracticingas a mission engineer might be asked a subset of these questions, while someone who isconsidered a thought leader aroundmission engineering, but is not currently practicing as amission engineer, might be asked a different question set. Finally, the RT-171 often askedfollow-upquestionstoprobeonwhatintervieweeshadsaid.Theseweredevelopedinrealtimeand though theyarenot listedbelow, theyare recorded in the team’s interview summaries.Thefollowing isasummaryofall interviewquestionsthatwereusedtopromptdiscussion inRT-171,groupedaccordingtothegeneralareaofinformationbeingexamined.MissionEngineering

• Pleasedescribehowyougotintomissionengineering.• Inyourownwords,whatismissionengineering?• Whatisyour[organization's]philosophyonmissionengineering?• Whatisyour[organization's]processorapproachtoperformingmissionengineering?

o Howeffectivedoyoufindthisapproachinpractice?


94

• Doyouseeanyoverlapintheactivitiesofsystemsengineeringandmissionengineering?

o Arethereelementsofsystemsengineeringthatareparticularlycriticalformissionengineering?

• Whatdoyouseeascriticalchallengesinmissionengineering?• Pleasedescribeyourcurrentposition.Whatareyourkeyresponsibilities?• Whatisthekeyvalueyouthatyouprovideinthisposition?• Whatisthemostcriticalthingyoudotobeeffectiveinyourcurrentposition?

Competencies

• Howdoyouidentify/selectthepeoplewhodomissionengineeringworkinyourorganization?

• Aretherecriticalskillsyoulookforwhenselectingmissionengineers?o Whyaretheseskillsimportantfromyourperspective?

[IfHelix/Atlasproficiencymodelisused.]• [ReviewofAtlasProficiencyModel]

o AreanyoftheAtlasproficienciescriticalforyourjob?• Whatskillsthatarecriticalformissionengineeringthataren’tincludedinAtlas?

• Earlieryouidentifiedsomekeychallengesformissionengineering.Whatskillsarecriticalinhelpingyoudealwiththesechallenges?

• Oftheskillsyouhaveidentifiedthatareimportantformissionengineering,whichofthesehasbeenparticularlyhelpfulforyoupersonally?Whichofthesecriticalskillsdoyouthinkaremostcommoninthecurrentworkforce?

• Whenyoulookatthepeoplecurrentlyperformingmissionengineeringinyourorganization,howwelldoyoufeeltheirskillsetsalignwiththemissionengineeringskillsyouhavedefined?

o Forareasofmisalignmentorgaps,doyouseewaysthatthiscanbeimproved?FutureDirections

• Whatisyourvisionforidealimplementationofmissionengineeringinyourorganization?

• Whatwouldhavetochangetomakethisareality?• Whatdoyouseeasthekeyriskstoyourorganizationdevelopingthemission

engineeringworkforceitwillneedfiveyearsfromnow?• Whataretheobstaclesorchallengesinobtainingthesecompetencies?

C.1.3 DATAANALYSIS(C)

Thefirststepindataanalysisistopreparesummariesofallinterviewsessions(C1).TheRT-171protocolsdonotincluderecording,sotheteammembersalltooknotes.Theteamdevelopedacomprehensivesummaryofeachinterview,whichparticipantsweregiventwoweekstoreviewand edit. Initial analysis, typically which provides a very high level coding of the dataset


95

provides insights into additional questions to ask or ways that questions might need to bereworded to improve the results received. (C2).Because it took longer thanexpected to findschedule some interviews, the team took advantage of the time delay by developingpreliminaryfindingsbaseduponthedataonhandandsettingupanalysistoolsandqueriessothatasadditionaldatawasgenerated, itcouldbeadded intotheexisting infrastructure (C3).Detailed qualitative and quantitative analyses, using software tools as necessary, have beenperformedallthedatathathasbeencollectedthroughRT-171interviews(C4).CodingThe interview dataset comprises summaries from 32 individuals. The RT-171 team usesqualitativedataanalysis,primarilythroughdatacoding.Codingis“asystematicwayinwhichtocondenseextensivedatasets intosmalleranalyzableunits throughthecreationofcategoriesand concepts derived from thedata.” (Lockyer 2004) Codes canbe layered, andevolveovertime,asexplainedbelow.Themaintypeofcodingdonebytheteamiscalled“opencoding”,the purpose of which is to break down, compare, and categorize data (Strauss and Corbin2014).Oneofthestrengthsofthecodingapproachisthatcodescanoverlap-individualsmaydiscussseveral issues togetherandresearcherscan layermultiplecodestogether.Notonlydoes thishelptogiveatruecharacterizationofthedata,butcommonpatternsinoverlapsmayprovideuseful insights.Additionalcodeswerethenaddedto thissubsetof thedata to furtherclarifythepatterns.Forexample:

Inthisexample,thereareseveralareasofcoding:

• Theskillhardesttofindisintheoperationscontext.Trainingdoesn’treallywork.o Thestatementthat“operationscontext”isadifficultskilltofindhighlightsthatit

isseenasavaluablecompetencyformissionengineers.• Theskillhardesttofindisintheoperationscontext.Trainingdoesn’treallywork.

o Thedifficultyoffindingindividualswithoperationalcontexthighlightsaspecificworkforceissueintheinterviewee’sorganization.

• Theskillhardesttofindisintheoperationscontext.Trainingdoesn’treallywork.o Thestatementthattrainingdoesnotworkhelpsprovideinsightintothecareer

paths for mission engineers. If this statement is true, then experiences inoperationalcontextarerequiredtogrowtheseskills.

By examining how often characteristics were cross-coded, it helped to identify relationshipsthatparticipantsbelievedareimportantacrossorganizations.Figure25providesanexampleofthecodingcomparisons.Thehigherthebar,thehighertheoverlapinexcerptsbetweencodes.This provides the team with insight into relationships between and patterns aroundcharacteristicsbasedonhowintervieweesdiscussedthem.

Theskillhardesttofindisintheoperationscontext.Trainingdoesn’treallywork.


96

Figure25.ExampleofCodingRelationships

The structure that emerged formission engineering includedmany areas, one ofwhichwascompetency.Initially,thecompetenciesaroseinthreenaturalgroups:technical,non-technical,and mindset-related. The team went through three levels of coding for each interview, asillustrated in Figure 26. The first pass identified all the portions of text (excerpts) thatwererelated to competencies. These excerpts were all reviewed to identify whether thecompetenciesdescribedweretechnical,non-technical,orrelatedspecificallytomindset.Theneachofthesesetsofexcerptswasanalyzedathirdtimetoidentifythespecificcompetenciesidentified.Figure24providesjustafewexamples:theseincludedeverythingfromspecificskillstounderstandingthecontextinwhchmissionengineeringoccurs.

Figure26.ExampleofLevelsofCoding

Competencies

Level1 Level2

Technical

Non-Technical

Mindset

SystemsEngineering

MissionConcept

Architecture

OperationalContext

Level3


97

The team then was able to drill down into the excerpts associated with each individualcompetency to identify how the competency is addressed, areas of concern, etc. And thecoding was used to perform basic analysis of the dataset. (See Section 1, Section 4, andAppendixE).SynthesisThe coding approach was critical to organizing and analyzing the data. To develop acompetency framework, the teamreviewedeachof the individualcompetencies identified ininterviews and look for patterns and natural groupings. This was informed by the literaturereview,andinparticulartheorganizationofothercompetencymodelsthatwerereviewed.ThisisdescribedinAppendixE.

C.1.4 ANSWERRESEARCHQUESTIONS(D)

The RT-171 team put a significant amount of effort into analyzing the interview data andcombining itwith the literature review results, todevelopapreliminaryunderstandingof anappropriate competency framework for mission engineering (D1). All analysis performed ondata collection was intended to answer the RT-171 research questions on both broad anddetailedlevels(D2).

C.1.5 PUBLISHRESULTS(E)

Publishing reports andpapers for public consumption is a keyobjective forRT-171 research.This report represents the key technical report publication for RT-171 (F1). All results andobservations reported in RT-171 publishing are done in an anonymous aggregated manner.NothingpublishedbyRT-171 istraceabletoanyparticular individualortoanorganization. Inaddition,peer-reviewedconferenceandjournalpapershavebeensubmittedforpublicationforwidedisseminationofRT-171results.(E2).AcompletelistofRT-171-relatedpublicationscanbefoundinAppendixA.

C.1.6 METHODOLOGYREVIEW(F)

Datacollectionandanalysisisbeingperformediteratively,asRT-171continuestoidentifyandvisitorganizations.Afteranysitevisitandbeforethenextone,areviewisconductedtoidentifyanyupdatestotheinterviewquestionsorprocess(F1).

C.2DATASET

Thecurrentdataset formissionengineeringcomprises interviewswith32 individualsandtheextensiveliteraturereview(seeAppendixD,belowfordetailsontheliteraturereview).Figure27showsthedistributionbyorganizationtype.


98

Figure27.PercentageofIntervieweesbyOrganizationType.

31%

28%

16%

13%

9%

3%

0%

5%

10%

15%

20%

25%

30%

35%

Navy Army DoD (General) Non-Government Non-DoDGovernment

Air Force

Percentage of Participants by Organization Type (N=32)

Organization Type


99

APPENDIXD:LITERATUREREVIEW

MissionengineeringisanewlyestablishedpracticeinmanyUSDoDandindustryorganizationsthat applies the mission context to system of systems and complex systems. Most currentsystems engineering practices do not fully address the unique characteristics of missionengineering,addressingtheend-to-endmissionasthe‘system’andextendingfurtherbeyonddataexchangebetweentheindividualsystemsforcross-cuttingfunctions,controlsandtradesacrossthesystems.Thissectionprovidesabackgroundofsystemofsystemsengineering(SoSE)tobetterunderstandmissionengineeringasSoSEisappliedinanoperationalcontext.

D.1 SYSTEMSENGINEERINGANDSYSTEMOFSYSTEMS:DEFINITIONANDSCOPE

Asystemisdefinedas“afunctionally,physically,and/orbehaviorallyrelatedgroupofregularlyinteractingorinterdependentelements;thatgroupofelementsformingaunifiedwhole”(DoD2008). SE is defined as a methodical and disciplined approach for the specification, design,development, realization, technical management, operations and requirement of a system.Traditional SE focuses on the development of systems with a stable architecture and statictechnologies. Under these principles, the manufacturing organization follows a verticalintegrationapproach,meaningthat ithascompletecontroloverall the interfacesneededfordevelopingthesystem.Itisalsoassumedthatallsystem’srequirementsareknownandcanbefrozen in time without affecting its performance (Azani 2008). Therefore, it is possible toanticipateandforecastthebehaviorofthesystematanystate.The 21th century is recognized as the inflection point in the design, development, andmaintenanceofengineeredsystems.The introductionof individualworkspacestoreplacebigcalculatorsopenedthedoor for lesscentralizedmanagementorganizations.Partnershipsandalliancesledtoanetworkoforganizationsthatshareknowledgeandresources,andinsteadofrelying in controlling theentire system lifecycle, theyhave transitioned from focusingon theproductionlinetoownershipofthedata(Luzeaux,2008).Also,evolvingstakeholderneedsandrapidchanging technologies result in the inabilityof thesystemtooperate inpredeterminedscenarios. Hence, systems are increasing in complexity due to: the participation of multiplestakeholders in the development of the system, their implementation in unanticipatedscenarios,andtorapidevolvingtechnologies(Azani,2008,Luzeaux,2008,Jamshidi,2009).Nowadays, systems engineers must deliver systems with self-organized and self-regulatedcapabilities. Systems must also be able to respond to evolving needs and rapid changingtechnologies(Azani,2008).Forinstance,tosupportcomplexoperationsaroundtheglobe,theU.S. DoD shifted from a user needs approach to a capabilities based approach (Dahmann,2008). The development of new capabilities relies in the combination systems, which mustwork together to meet the end objective. However, traditional SE practices are fixed andassumethatsystemsdonotevolveovertime.Therefore,newstrategiesandapproachesthatconsidertheincreaseincomplexityareneeded.


100

Currently,theUSDoDcapabilitiesareprovidedbyanaggregationofsystems,alsoreferredtoasSoS.AccordingtotheUSDoD,thedefinitionofanSoSis“asetorarrangementofsystemsthat results when independent and useful systems are integrated into a larger system thatdelivers unique capabilities (DoD, 2004). There are four types of SoS classifications that arerequired to understand and characterize the SoS based on the relationships of the systemswithintheSoS[DahmannandBaldwin,2008]:

• Directed SoS, where the integrated system-of-systems is built andmanaged to fulfillspecific purposes. It is centrally managed during long-term operation to continue tofulfillthosepurposesaswellasanynewonesthesystemownersmightwishtoaddress.

• Virtual SoS, which lack a central management authority and a centrally agreed uponpurposeforthesystem-of-systems.

• Collaborative SoS,where the component systems interactmore or less voluntarily tofulfillagreeduponcentralpurposes.

• Acknowledged SoS, which have recognized objectives, a designated manager, andresources for the SoS; however, the constituent systems retain their independentownership,objectives,funding,anddevelopmentandsustainmentapproaches.

There is a focus on acknowledged SoS in the US DoD as the warfighter capabilities areincreasinglyemphasized.UsuallyintheUSDoD,multiplesystemsarerequiredtomeettheend-to-endcapabilityneeds.WhenaformalSoSisrecognizedbasedonitsneed,anorganizationisrecognized as being responsible for the SoS area, including the general definition of theobjectiveoftheSoS.ButthisusuallydoesnotincludethechangesinownershipofthesystemsintheSoSorchanges intheobjectivesoftheconstituentsystems. Also,thecurrentsystemsarestillinuseandneededforitsoriginalintent.Thispresentsachallengeespeciallybasedonthehierarchicalstructuresofthedefenseacquisitionmanagement.

D.2 SYSTEMOFSYSTEMSENGINEERING:DEFINITIONANDSCOPE

SystemofSystems(SoS)systemsengineering(SE)“dealswithplanning,analyzing,organizationandintegratingthecapabilitiesofnewandexistingsystemsintoanSoScapabilitygreaterthanthesumofthecapabilitiesofitsconstituentparts(DoD,2004).Withoutamissionoperationalcontext, there is a lack of direction or driving force to aid the developers andmanagers indeterminingwhichsystemshavetobeinvolved,whatfunctionstheyhavetoperform,andhowthe operators or users will make use of these systems. This research effort provides theadditionalvalueoffacilitatinganimprovedunderstandingoftheincreasingcomplexitiesinSoS.

According tonumerous sources, thedifferencesbetween systemsand systemsof systemsasapplied to systems engineering were considered based on management and oversight,operational focus, implementation, and engineering and design considerations, as shown inTable12(DahmannandBaldwin2008andNeagaetal.2009).


101

Table12:ComparisonbetweenSystemsandSystemsofSystemsasAppliedtoSystemsEngineering(DahmannandBaldwin2008andNeagaet.al.,2009)

SystemsEngineering(SE) SystemsofSystemsEngineering(SoSE)ManagementandOversight

StakeholderInvolvementClearsetofstakeholders Multiplelevelsofstakeholderswithmixedand

possiblycompetinginterests

GovernanceAlignedmanagementandfunding

AddedlevelsofcomplexityduetomanagementandfundingforbothSoSandsystems;SoSdoesnothavecontroloverallconstituentsystems

OperationalFocus(Goals)

OperationalFocusDesignedanddevelopedtomeetcommonobjectives

CalledupontomeetnewSoSobjectivesusingsystemswhoseobjectivesmayormaynotalignwiththeSoSobjectives

Implementation

Acquisition/Development

Alignedtoestablishedacquisitionanddevelopmentprocess

Crossmultiplesystemlifecyclesacrossasynchronousanddevelopmentefforts,involvinglegacysystems,developmentalsystems,andtechnologyinsertion

Process Well-established Learningandadaptation

TestandEvaluation(T&E)

Testandevaluationofthesystemispossible.SystemrequirementsdrivethesystemT&EanduseMeasuresofPerformance(MoP)

Testingismorechallengingduetosystems'asynchronouslifecyclesofcomponentsystemsandthecomplexityofalltheparts.AttheSoSlevel,MeasuresofEffectivenessareneeded,whicharedifficulttodefine,aswellasMoPs

EngineeringandDesignConsiderations

BoundariesandInterfaces

SystemofInterest(SOI)isdefinedbyfocusingonboundariesandinterfaces

ThedynamicandreconfigurablenatureofSoSmeanthatboundariesareinterfacesmaychange.Also,componentsystemsmaybelongtomorethanoneSoSandhavevariableavailability

PerformanceandBehaviorPerformanceofthesystemtomeetperformanceobjectives

PerformanceacrosstheSoSthatsatisfiesSoScapabilityobjectiveswhilebalancingneedsoftheconstituentsystems

MetricsDerivationfromrequirementsisstraightforward

Difficulttodefine,agree,andquantifyduetoindependentmanagementofcomponentsystems

The initial key SoS SE artifacts were identified with the purpose of developing a set ofapproachestoapplySEtoSoSbyunderstandingtheuseoftheSEartifactsandhowtheywereusedinSoSSE(Dahmannetal.,2010).InordertoapplySEtoSoS,thecharacteristicsofSoSandtheirimpactontheSEprocessesneedtobeunderstood.BasedontheUSDoDSEGuideforSoS,the SoS systems engineers focus on the core elements listed below asmultiple existing andemergentsystemsareevolvingandassembledtomeetthecapabilityobjectives:

“In SoS SE, systems engineers are key players in the core elements of: (1) translating SoScapabilityobjectives into SoS requirements, (2) assessing theextent towhich these capabilityobjectives are being addressed, and (3) monitoring and assessing the impact of external


102

changesontheSoS.CentraltoSoSSE is: (4)understandingthesystemsthatcontributetotheSoSandtheirrelationships,(5)developinganarchitecturefortheSoSthatactsasapersistentframework for (6) addressing SoS requirementsand solutionoptions. Finally, the SoS systemsengineer(7)orchestratesenhancementstotheSoS,whilemonitoringandintegratingchangesmadeinthesystemstoimprovetheperformanceoftheSoS.”(U.S.DoDSEGuideforSoS,2008)

Figure28 illustratestheSoSSEartifact thatwasdevelopedaspartofan InternationalSoSSEproject under The Technical Cooperation Program (TTCP). Figure 28 is also described as atrapezemodel that provides a conceptual view of SoS SE. However, it is not very useful topractitionerstohelpthemdevelopanimplementationapproach.

Figure28:ArtifactsintheContextofCoreElementsofSoSSE(Dahmann2010)

Therewasan initiative todevelop thewavemodelbasedon theSoSSEartifact. In thewavemodel,themainSoSactivitiesarethefollowing:a)SoSAnalysis,b)SoSArchitecture,c)Planforupdate,andd) ImplementSoSupdatefor(Dahmann2012).TheSoSwavemodel inFigure29wasdevelopedto“unwind”thetrapezemodelandprovideanintuitiveviewofSoSSEintermsofsuccessionsofmajorsteps in implementinganSoSSEprocess. ThisSoSSEwavemodel isbuilt based on ‘wave planning’ for complex systems management by Dombkins. (Dombkins2007andDahmannetal.2011)


103

Figure29:SoSWaveModel(Dombkins2008andDahmannetal.2011)

AstheSoSEwavemodelisappliedtomissionengineering,themajorstepsinimplementinganSoSEprocessaredescribedinTable13(Dahmann,et.al.,2018).ThecoloredboxesinTable13correspond to the respective elements in the SoSEwavemodel in Figure 29. This literaturereview finding of the criticalmission engineering activities correlate to the research findingsfromtheinterviewparticipants,asreportedinSection2.2.5.

Table13:SoSEWaveModelAppliedtoMissionEngineering(Dahmannet.al.2018)

ConductSoSAnalysisDefine the mission including mission threads and mission context (includingmissionobjectives,CONOPS,scenarios,keyfunctionality,threat)Identifycurrentsystemssupportingthemissionandhowtheyareemployed(howisthemissionimplementednow?)AssessmissionperformancetoassesshowwellcurrentsystemsworktogethertomeetthemissionobjectivesIdentifygapsfromamissioneffectivenessperspectiveandfaultisolatethesourceofgaps

DevelopSoSArchitecture Identify and assess options for improving the mission effectiveness (includingchanges on how the systems are employed as new or different systems, systemsupdatesandnon-materialconsiderations)

PlanSoSUpdate Guide systems acquisitions, from requirements through implementation to testandmaintenancetoassureeffectivemissionexecution

ImplementSoSUpdates ConductmissionlevelintegrationandtestingMonitor mission effectiveness with changes in mission context, scenarios andthreatcapabilities

Due to the distinctive characteristics of SoSE, there are implications to the SoS testing andevaluation(T&E).Dahmannet.al.(2010) reviewedtheSoScharacteristicsastheyimpacttheT&E,aswellashowitwasbeingaddressedintheSoSEcommunityofpractitioners.Intermsofthe implications of the SoS T&E on themanagement and oversight of SoS, Dahmann et. al.(2010) found it more difficult to establish the validation criteria and also harder for thegovernancetoexplicitlyimposeSoSconditionsonsystemT&E.TheoperationalenvironmentofthesystemlevelmaynothaveaclearequivalenceinSoSconditionsthatrequireT&E.Fortheimplementation aspect, the SoS T&E depends on testing of the constituent systems to SoS


104

requirements,andextendstowardsSoSleveltesting.Itmaybehardtobringmultiplesystemstogether in synchrony across the capability evolution for T&E. Finally, for engineering anddesign considerations, theremay be increased subjectivity in assessing the performance andbehavioroftheSoSandadditionaltestpointsmayberequiredtoconfirmthebehavioroftheSoS(Dahmannet.al.,2010).Since this study focused on critical mission engineering activities, development planning isconsidered as one of the key mission engineering tasks. Development planning is “theengineeringanalysesandtechnicalplanningactivitiesthatprovidethefoundationforinformedinvestmentdecisionson thepathamaterieldevelopment follows tomeetoperationalneedseffectively,affordablyandsustainably”(DAG,2017).ConsideringthatmostSoSdevelopmentisevolutionary, continuousmonitoring is requiredandchangeswillbemadeasupdateson theSoS, as shown in the SoSE wave model. The mission context encompasses a complexenvironmentthatrequiresstrategicdevelopmentplanningthatmayevolveovertimebasedonthecapabilityneedsandchangesinsystem.

As themission-focuson SoSeffortshaveemerged in thepast five to six years, theU.S.DoDengineeringcommunitycannowsuccessfullyassessanddeterminewhichsystemsarerelevanttoacapabilityandhowtomodifythesesystemstosupportcriticalmissionareassuchascloseairsupport,ballisticmissiledefense,andsatellitecommunications.Toaddresstheapplicationof SE to SoS, the DoD Guide to SE for SoS (2008) and the International Organization forStandards/ International Electrotechnical Commission/ Institute of Electrical and ElectronicsEngineers(ISO/IEC/IEEE)15288,AppendixGareusedasguidelines.

D.3 CAPABILITIESENGINEERING

D.3.1 PERSPECTIVES

The termcapabilityis widely used acrossmany industrial sectors and has begun to take onvarious specific meanings across, and even within, those sectors. Terms such as capability-based acquisition, capability engineering and management, life capability management,capability sponsor, etc. arenowubiquitous indefenseandelsewhere.Henshawet al. (2011)haveidentifiedatleasteightworldviewsofcapabilityandcapabilityengineeringandconcludedthat the task of capability engineering is not consistently defined across the differentcommunities.Whilstmostpractitionersrecognizethatthere isastrongrelationshipbetweencapabilityandsystemofsystems(SoS),thereisnoagreeduponposition.However,therearetwobeliefsthatarewidelyacceptedamongthedifferentcommunities,including:

• a capability comprises a range of systems, processes, people, information andorganizations. (i.e. a system at levels three through five in Hitchin's (2003) five layermodel,suchasaCarrier-Strikecapability)and

• the capability is an emergent property of SoS (i.e. the capability of Carrier-Strike toengagetargetswithin300milesofthesea.)


105

D.3.2 SERVICESVIEWOFSYSTEMOFSYSTEMSENGINEERING

TheGuide to the Systems Engineering Body of Knowledge (SEBoK) has the following to sayaboutservice-orientedsystemsofsystems(BKCASEAuthors2017):

Asystemofsystems(SoS)istypicallyapproachedfromtheviewpointofbringingtogethermultiple systems toprovidebroader capability. Thenetworkingof theconstituentsystemsinaSoSisoftenakeypartofanSoS.Insomecircumstances,theentire contentofaSoS is informationand theSoSbrings togethermultipleinformationsystemstosupportthe informationneedsofabroadercommunity.These information technology(IT)-based SoS’s have the same set ofcharacteristicsofotherSoS’sandfacemanyofthesamechallenges.Currently,IThas adopted a ‘services’ view of this type of SoS and increasingly applies aInternational Organization for Standardization (ISO) 20000 series (Informationtechnology -- Service management) or Information Technology InfrastructureLibrary(ITIL)v.3(OGC2009)basedapproachtothedesignandmanagementofinformation-basedSoS.AserviceperspectivesimplifiesSoSEasit:

• isamorenaturalwayforuserstointeractwithandunderstandaSoS,• allowsdesignerstodesignspecificservicestomeetdefinedperformanceandeffectivenesstargets,and• enablesspecificservicelevelstobetestedandmonitoredthroughlife.Althoughithasnotbeenproventobeuniversallyapplicable,theservicesviewworkswellinbothITandtransportationSoS.

D.4 RELATIONSHIPBETWEENSYSTEMOFSYSTEMSANDMISSIONENGINEERING

Allmodelsarewrong,butsomeareuseful.–GeorgeBoxAs language isbutamodel forreality, it is limited in itsprecisionandquiteoftenusefulness.Themeaningoflanguageiscriticallydependentuponitscontextandthelensoftheonewhousesit.Creatingaspecificdefinitionorevendescriptionofaphenomenonthatisinterpretedinexactlythesamewaybyallobserversmaynotbepossible. Onecouldarguethatit isonlyinmathematicsthatthisispossible,andonlypossiblewhenthemathematicsarenotintendedtocorrespondwithourphenomenologicalreality.Thus,itisnotsurprisingthatthereisconfusionaround the meaning of terms such as “systems of systems” and “mission engineering”.However,itispossibletogainanunderstandingoftherolesthatthesetermsmayplayinhowpeopleunderstandvariousphenomenon.Thosewhointendtoconceive,design,buildandusesystemsgenerallydosofor(a)apurpose,and (b)with the intention for these systems towork in certain contexts (which includes thesystemitself).Itisdifficulttoimagineasystemsdiscussioninwhichbothofthese,thesystembehaviorandthesystemcontext,arenotrelevant.Quiteoften,thosewhoareonlyawareofasinglecontextmaylosesightoftheboundariesandlimitsofsuchacontext,muchthebestway


106

tounderstandacultureoftencomesfromtheobservationsofthosewhoarefromadifferentculture.Frequently,misunderstandingmayoccurduetodifferencesintheunspoken,perhapsunappreciated, differences in context. Therefore, itmightbeof value to consider the terms‘Systems of Systems’ and ‘Mission Engineering’ qualitatively with respect to the notions ofpurposeandcontext.Figure 30 constructs a space with increasing complexity in context in the vertical axis andincreasingcomplexityineffectinthehorizontalaxis.Onecanusethistaxonomytoanalyzetheconstructionandbehaviorofsystemsovertimeinanumberofdomains.Clearly,systemsthataresimpleinnaturewithsimpleexpectedbehaviorsarewell-understoodandsubsequentlyarenot of much interest in this discussion. With limited expectations in use and construction,systemscanbethoughtofassimple. Systemsget interesting,which istosayunpredictable,whentheexpectationsofcontextandusebecomemorecomplex.As an example, consider the automobile industry. The expected outcome is to design,development, manufacture and sell automobiles at a profit. To reduce uncertainty, and tomaximizeprofitatthetime,FordMotorCorporationbecameaverticallyorientedcompanyanddid everything from mining the ore, to making the steel, to building and selling the cars.3However, over time, this became a far less successful strategy economically as singlecompanies were not capable ofmaintaining technical or economic leadership is such a vastarrayof supporting technologies.As a result, theoverall context for the systemcreating thecarsbecameevermorecomplex.Beingsuccessfulintheautomobileindustryinvolvedbeingamasterofthesupplychain.Whileonecouldarguethatthisprovidedfarlesscontroloverthe‘system’, it supported a marketplace economics that enabled far greater capabilities. Theautomobile industry, and the exquisite supply chain with its distributed control, became a‘systemofsystems’inthatnooneentityhadthelevelofcontrolofanintegratedindustry.OnFigure30,thisisshownasthemoveupfromthelower,farrightquadrant.A very different example is the often-referenced Apollo program. In this case, the desiredoutcomeisnotaproduct,butratherisasingularoutcome,thatputtingamanonthemoonandreturning him home safely before the end of the decade. As thiswas seen as amission ofexistential importance,theApolloprogramhadagreatdealofcontroloveritsoperations. Infact, it almost single-handedly created a new industry in micro-electronics and drove thedevelopment of numerous critical technologies.4 The designers of Apollo determined therequirementsofthesystem,thesubsystemsanditscomponentsanditwasuptothesuppliers(aswellastheastronauts)tosupportthem.ApollocanbeshowninFigure30asamovementtotherightfromthelowerleftquadrant.Defenseactionsandcampaignscanbeseeninmuchthesameway,wherethecomplexitycanbeseenintherangeofthedesiredeffects.However,inthemilitary,therangeindesiredeffectswasoftensupportedbythehumanelementasthe3 http://www.economist.com/node/13173671, 3/27/2009,Moving on up, Is the recession heralding a return toHenryFord'smodel?4 https://www.computerworld.com/article/2525898/app-development/nasa-s-apollo-technology-has-changed-history.html,July20,2009,NASA'sApollotechnologyhaschangedhistory,Apollolunarprogrammadeastaggeringcontributiontohightechdevelopment.


107

glue that could be used to tie all of the pieces together. For example, in the navy thecommandersofvesselscouldmakedecisionsthatwerecommunicatedacrossthefleetthroughflagmenandotherhumanelements.However,asmoreoftheseoperationsaresupportedbytechnology, thesystemsmissions requirementsneed tobemorecarefullyplanned for futureoperations.Whatisinterestingisthatbothcommercialindustryanddefensearemovingtothequadrantintheupperrightwhich isoneofhighcomplexityboth inmissionand incontext. For industry,the changing notion of systemswith increased autonomy and virtualization has dramaticallyincreasedthe importanceofdeliveredservices. Forexample, intheautomobile industry, theincreaseduseofautonomymaymakethedeliveryofanautomobileobsoletewhentheserviceofmobilitybecomesfarmorerelevant.Thenotionofautomobileownershipmaybecomefarless important as the automobile becomes a ‘component’ in the mobility system ofautonomous Uber and Lyft. Likewise, it is no longer possible for the military to be thepredominant force in technology development as was done with Apollo, or even to havededicatedcapabilitiesforallofthismissionsandforces.Thenetresultisthecontextofsystemsofsystems inwhichcontrol is tradedoff for increasecapabilitiesata lowercost. Thus,bothcommercialindustryanddefensewillneedtomastercomplexityinbothofthesedimensions.Rather than attempting to determine the differences between ‘systems of systems’ and‘missionengineering’,thisdiscussionshowshowthesetermscanbeusedtodescribecontextandrangeofintent.Makingthesedistinctionscanprovidethemeanstodiscussthechallengesand concerns in each of the quadrants shown in Figure 30, and allow for the analysis anddiscussionofdecisionsandactionsthroughoutthisspace.


108

Figure30.RelationshipbetweenMissionEngineeringandSystemofSystemsEngineering


109

APPENDIXE:INTERVIEWRESULTSONCOMPETENCY

Asdescribed in themethodology, the teamcoded thedataseveral times todevelopa roughstructureofcompetencies. In total, therewere1834excerptsaroundcompetencyacross thedataset,ashighlightedinFigure30.Everyindividualwhoparticipated(N=32)discussedmissionengineeringcompetenciesaspartoftheirinterviews:

• 100%ofinterviewees(32)describedtechnicalcompetencies

• 94%ofinterviewees(30)describednon-technicalcompetencies

• 91%ofinterviewees(29)describedmindset-relatedcompetencies

ThebreakdownofindividualexcerptsintothesegroupscanbefoundinFigure31.

Figure31.PercentageofCompetencyExcerptsaroundSpecificTypesofCompetencies(N=1834)

E.1TECHNICALCOMPETENCIES

There were a total of 47 individual technical competencies identified across 1106 excerpts,ranging from competencies such as “risk management” to “mission design” to “domainknowledge”.Once thesewere identified, the team reviewed the competencies to determineappropriategroupingsforthecompetenciesaswellasanyappropriatecut-offs.Forexample,ifa competency was described only once by a single interviewee, and does not align withcompetencies found in related competencymodels, the team deemed that it did notmakesensetoincorporateitintotheMEcompetencymodel.


110

The three groupings for technical competencies – which align with competency modeldescribedinSection3:

• DisciplineandDomainFoundations

• MissionConcept

• SystemsEngineeringSkills

Because the co-coding approach means that several competencies might be addressedtogether,thereareatotalof1106excerptsfortechnicalcompetencies,butonly847ofthesewereunique(i.e.259oftheexcerptswerecodedacrossmultiplecompetenciesand,therefore,arecountedmorethanonceinthetotalexcerpts).Figure32showstheaggregationofthe847unique excerpts into the competency areas reflected in the ME Competency Framework(Section3).

Figure32.TechnicalCompetenciesExerptsAggregatedintoCompetencyAreas(N=847)

Eachofthesecompetencyareas–andtheskillsassociatedwiththemisdescribedinadditionaldetailbelow.

E.1.1DISCIPLINEANDDOMAINFOUNDATIONS

Many of the technical competencies described foundational skills, which provide missionengineersagroundingintechnicalskillsthatenablethemtoactastechnicalleadersinmissionengineering. Figure 33 provides an overview of the competency categories identified fromthesecompetenciesasreflectedintheMECompetencyFramework(Section3).


111

Figure33.DisciplineandDomainCompetencies(N=32interviewees;214excerpts)

Notethatsomeofthesecategoriesrepresentarecombinationofindividualcompetencies.Forexample,“EngineeringDisciplines”isacombinationof3individualcompetenciesdescribedbyinterviewees:

• Engineeringdisciplines(whichweredescribedeithergenerally,i.e.“havingafoundationinanengineeringdisciplineisimportant”orhighlightingaspecificdiscipline,i.e.“electricalengineeringisreallyimportantforourmission”)

• Breadthacrossdisciplines(e.g.itiscriticalformissionengineerstobeabletounderstandenoughoftheengineeringdisciplinestobeabletotranslatemission-relatedconcernstoengineersortounderstandtheengineeringsolutionwellenoughtounderstandthepotentialimpact(s)onamission)

• Appropriatedepth(e.g.havingenoughdepthinanengineeringdisciplinetohavecredibilityandgeneraltechnicalunderstanding)

Eachoftheseisrelatedtoviewthatthereisacriticalfoundationinanengineeringdisciplineforamissionengineer.ForthepurposesoftheMECompetencyFramework,thesewerecombined,thoughthebalanceofbreadthanddepthisdescribedintheFrameworkitself.“Problemsolving”washighlightedasan importantcompetency(citedby21%of inteviewees,butinlessthan5%oftheexcerptsontechnicalcompetency).Problemsolvingwascitedinthedataasanenabler todealingwithengineering, technology, and systems issues.Because this


112

representedasmallsetofthedataanditwasdescribedasanenablertoothercompetencies,itisnotexpresslyhighlightedintheMECompetencyFramework.Likewise, “rigor”, “change management” and “cost analysis” were cited, but each wasmeantionedbyonly1participant.Becauseofthese,theyarereflectedinthedataanalysisbutdidnotrisetothethresholdofbeingincludedintheMEComptencyFramework.This analysis gave rise to the firstMECompetencyArea:DisciplineandDomainFoundations,whichhassixcategories,asdescribedinSection3:

1. PrincipleandRelevantDisciplines

2. RelevantDomains

3. SystemCharacteristics

4. RelevantSystems

5. RelevantTechnologies

6. AcquisitionContext

E.1.2MISSIONCONCEPT

All of the interviewees described critical competencies that the team has identified in theMissionConcept area. In total, therewere 235 excerpts that described critical competenciesrelatedtothemissionconcept,asreflectedinFigure34.By far, themostcommonly-citedcategory in thisarea is relatedtobeingabletounderstand,work within, and describe the impacts on a mission from operational context. Operationalcontextisthecombinationoftheconditions,circumstances,andinfluencesthatwilldeterminewhichsystemswillbeused.Thiswascitedby90%ofintervieweesandmadeupover60%ofthetotalexcerptsrelatedtoMissionConcept. It isworthnotingthat, ingeneral,thiscompetencytendedtocomefromformeroperators,ratherthanbytrainingengineersinoperations.Asoneinterviewee explained, “We have a mix of operators and engineers. It’s really importantbecausetheygiveandtakepotentialtechnicalsolutions,andhavetheopportunitytointeractwith thesolutions,how it’sautomatedornot,how it’sneededversushow itoperates.”ThiswasacommonmodelforincorporatingoperationalcontextintoMEteams.


113

Figure34.MissionConceptCompetencies(N=32interviewees;235excerpts)

TheMissionCONOPSistheviewofthecriticalsystemsrequiredtocompleteamissionwhichhighlightshowthesesystemswillinteractatahigh-leveltoproducethedesiredmissioneffects.Thiswascitedbyoverthree-quartersoftheinterviewees(78%),thoughwasnotmentionedasfrequently as the operational context. Mission Scenarios/Threads define the end-to-endexecution of amission and enable individuals to understandhowall the systemsof systemswork together, and were also cited by the majority (56%) of the interviewees. The finalcategoryinthisareaisDOTMLPF–anacronymthatstandsforDoctrine,Organization,Trainingl,Materiel,Logistics,Personnel,Facilities.Inthiscontext,thiscompetencydescribestheabilitytodefine a mission not only by the systems involved but also across the non-materiel space,includinghowitcouldbeused,howitmightoperate,howitcouldbesupported,etc.ExcerptsabouttheDOTMLPFcompetencydescribedhowcriticalitisthatmissionengineerslookacrossthespaceanddonotfocussolelyonmaterielsolutions.

E.1.3SYSTEMSENGINEERINGSKILLS

Itshouldnotbesurprising,giventhestrongrelationshipintheinterviewdatabetweenmissionengineeringandsystemsengineering,thatsystemsengineeringskillswerefrequentlycitedascritical formission engineers to be successful. Therewere a total of 380 excerpts regardingsystemsengineeringcompetenciesinthestudy(covering45%ofthetotalexcerptsontechnicalcompetencies). Figure 35 provides an overview of the various systems engineeringcompetenciesdescribedintheinterviews.


114

Figure35.SystemsEngineeringCompetencies(N=32interviewees;380excerpts)

Because therewere22 individualcompetencies identified, tomakecutoffdeterminations forsystemsengineeringskills,thefirstmetricwasthepercentageofintervieweeswhodescribedaspecific capability. For instance, while agilemethods, critical tools, and prototypingwere alldescribed, theywere each described by only a single individual in the sample,meaning it isreasonable to leave them out of theME Competency Framework. This does notmean thattheseskillsareunimportantbutthatperhapstheyaremorecrucialincertainorganizationsorforaveryspecificmission.Competencies included in the ME Competency Framework related to systems engineeringincluded:

1. SoSEngineering

2. Analysis

3. Architecture

4. ModelingandSimulation

5. Requirements

6. Integration

7. GapAnalysis

SoSengineeringwasthemostcommonly-citedskill(91%ofintervieweesand16%ofexcerpts)andemergedintwoways:eitherindividualsspecificallycitedSoSengineeringasacriticalskillor individuals cited “systems engineering” as a critical skill, but they had defined “mission”


115

systems as SoS’s and had stated that the SoS perspective was critical. Likewise, Analysis,Architecture,ModelingandSimulation,andRequirementswereallheavilycitedasimportant.ThecutoffforinclusionintheSystemsEngineeringareawascompetenciescitedbyatleast34%oftheinterviewees,whichincorporatedIntegrationandGapAnalysis.However,itisimportanttonotethattheMECompetencyFrameworkisintendedtobetailoredand, to thatend, it is likely that theSystemsEngineeringareawouldbe tailored tohighlightcrucialskillsforcertaintypesofsystems,operationalcontexts,etc.

E.2SYSTEMSMINDSET

The Systems Mindset area is primarily focused on patterns of thinking, perceiving, andapproachingataskthatareparticularlyrelevanttomissionengineers.Ofthe32interviewees,91% (29 individuals) described the importance of the systems mindset. As part of themethodology, theAtlasproficiencymodelwasprovidedasa readaheadto the interviewees.Interestingly, the conceptof “systemsmindset” resonatedwithmanyof these individuals. Infact,45%oftheexcerptsaroundsystemsmindsetwerespecificallyreferencingtheAtlasmodel;thesemadeup45%ofthetotalexcerpts.

Figure36.SystemsMindsetCompetencies(N=29interviewees;157excerpts)

Bigpicture thinking–which includes theability to stepbackand takeabroader viewof theproblemathand–was citedas importantby76%of the interviewees. The competenciesoffutureprojection and innovation and creativitywerehighlightedbyonlyone interviewee, so


116

were not incorporated into the competency framework. However, if additional data werecollectedandaddedtothedataset,thiscouldchange.Competencies included in the ME Competency Framework related to systems mindsetincluded:

1. BigPicture

2. Adaptablity

3. ParadoxicalMindset

4. Abstraction

5. CriticalThinking

E.3“NON-TECHNICAL”COMPETENCIES

These skills, which might also be referred to as “soft skills” are nonetheless critical for thesuccesses of mission engineers. These competencies were grouped into two areas:InterpersonalSkillsandTechncialLeadership.Intotal,30individuals(94%)describedtheseskillsas critically important formission engineering. Therewere a total of 411 excerpts regardingthese “non-technical” competencies, which included overlaps in coding between relatedcompetencies.

E.3.1INTERPERSONALSKILLS

For interpersonal skills, 30 individuals (94%) provided 199 individual excerpts on theimportanceofinterpersonalskillsformissionengineers.Figure37providesanoverviewofthecategoriesinthiscompetencyarea:

1. Communication

2. Translation

3. EnterpriseContext

4. Building&UtilizingaSMENetwork

5. Coordination

6. Influence,Persuasion,andNegotiation


117

Figure37.InterpersonalSkillsCompetencies(N=30interviewees;199excerpts)

The first two categories were described by over two-thirds of the interviewees. In fact,translation is a very specific type of communication, in which the concerns of a givenstakeholder are “translated” into the language of another stakeholder. The team initiallyconsidred combining these, but the data was divided into these two groups: generalcommunication skills and this specific idea of translation. Because 70% of the intervieweesspecificallycitedtranslation, itwaskeptasaseparatecategory, though it isofcoursecloselytiedtocommunication.

E.3.2TECHNICALLEADERSHIP

For Technical Leadership, 30 individuals (94%) provided 139 excerpts that described thesecriticalskills.Figure38providesanoverviewofthedifferentcompetenciesidentifiedandhowtheyhavebeengroupedintocategoriesfortheMECompetencyFramework:

1. GuidingDiverseStakeholders

2. TeamBuilding

3. PoliticalSavvy

4. DecisionMaking


118

5. WorkforceDevelopment

Figure38.TechnicalLeadershipCompetencyCategories(N=30individuals;139excerpts)

Theguidanceofdiversestakeholdersemergedasaclearthemeinthedata,withtwo-thirdsoftheintervieweescitingthisasacriticalskill.Teambuilding,politicalsavvy,decisionmaking,andworkforce development also emerge. The team considered not including workforcedevelopment, as this is required in any discipline, but most interviewees who discussedworkforcedissuesdescribedasituationinwhichtheywerenotsufficientlystaffedtoperformMEandweresimultaneouslytryingtoperformMEwhilegrowingnewMEs.Asthisaddressesacriticalchallenge,itwasincludedbytheteam.Projectmanagement(PM)wascitedby17%ofthe intervieweeswhodiscussed technical leadership,but60%of these individualswere fromthe same organization. Therefore, itwas determined that thismay be critically important insome contexts, but does not rise to the level of individual framework. Likewise, becauseculture, a providing a business case forME, and the social scienceswerementioned by fewinterviewees, they may be important in some contexts but were not included in the MECompetencyFramework.


119

APPENDIXF:EXISTINGACADEMICPROGRAMSINMISSIONENGINEERING

Table 14 provides a listing of all programs that have mission engineering related curriculaidentifiedbytheRT-171team.

Table14.AcademicProgramswithMissionEngineering-RelatedCoursesUniversityName CourseName CourseDescriptionAirForceInstituteofTechnology

SystemofSystems TheSystemofSystemscourseprovidesanoverviewofhowsystemsengineeringprocesselementsareappliedtotheplanning,developmentandfieldingofasystemofsystemscapability.Thecourseassumesbasicunderstandingofsystemsengineeringprocesselements.

CaliforniaInstituteofTechnology

SystemofSystems Thisprogramisdesignedtoprovideparticipantswithafullunderstandingofthechallengesofengineeringandintegratingsystemofsystems.Attheconclusionofthiscertificateprogram,participantswillunderstandtheapplicationofprocesses,methodsandtechniques(andtheirextensions)totheuniquechallengesofengineeringandintegratingsystemofsystems.

CommonwealthGraduateEngineeringProgramUniversities:GeorgeMasonUniversity,OldDominionUniversity,UniversityofVirginia,VirginiaCommonwealthUniversity,andVirginiaTech

SystemofSystemsEngineering

ComprehensivetreatmentofSystemofSystemsEngineering(SoSE),including;fundamentalsystemsprinciples,concepts,andgoverninglaws;complexandsimplesystems;underlyingparadigms,methodologiesandessentialmethodsforSoSEanalysis,design,andtransformation;complexsystemtransformation;currentstateofSoSEresearchandapplicationchallenges.Explorestherangeoftechnological,human/social,organizational/managerial,policy,andpoliticaldimensionsoftheSoSEproblemdomain.

JohnsHopkinsUniversity

SystemofSystemsEngineering

Thiscourseaddressesthespecialengineeringproblemsassociatedwithconceiving,developing,andoperatingsystemscomposedofgroupsofcomplexsystemscloselylinkedtofunctionasintegralentities.Thecoursewillstartwiththeunderlyingfundamentalsofsystems'requirements,design,testandevaluation,anddeployment,andhowtheyarealteredinthemulti-systemenvironment.Thesetopicswillthenbeextendedtoinformationflowandsysteminteroperability,confederatedmodelingandsimulation,useofcommercialoff-the-shelfelements,andsystemsengineeringcollaborationbetweendifferentorganizations.Advancedprinciplesofinformationfusion,causalitytheorywithBayesiannetworks,andcapabilitydependencieswillbeexplored.Severalcasestudieswillbediscussedforspecificmilitarysystemsofsystems,includingmissiledefenseandcombatantvehicledesign,aswellasselectedcommercialexamples.


120

UniversityName CourseName CourseDescriptionNavalPostgraduateSchool

SystemofSystemsDesignandDevelopment

ThiscoursediscussesthespecialproblemsofmanagingandengineeringsystemofsystemsfromtheLSIperspective.TopicsincludecharacteristicsofSoSintheLSImanagementenvironment,engineeringimplicationsofSoSissues,managementandengineeringmethodologyofSoS,SoSarchitecture,analysisofSoS,andtoolsforengineeringandmonitoringSoS.ManagingtheintegrationofSoSthroughanLSIrequiresattentiontothemeta-systemsimplicationsofchangesatthesystemslevel.ThiscoursediscussesthespecialproblemsofmanagingtheintegrationofsystemofsystemsfromtheLSIperspective.TopicsfromtheLSIperspectiveincludethecharacteristicsofthelargescaleSoS,programmanagementofSoSintegration,usesofSoSdesignandarchitecturefordecisionanalysis,feasibilityanalysisandapproachesforSoSintegration,SoScontractmanagement,andexecutionforSoSacquisitions.

PurdueUniversity SystemofSystemsModelingandAnalysis

Thegoalforthiscourseistoenablestudentstocharacterize,abstract,model,simulate,andanalyzeaspecialkindofsystemtermedasystem-of-systems(SoS).Thecoursewillcoveraselectfewtopicsindetail,butalsoexposestudentstointerestingareasoffurtherstudyandhighlighttheimportanceofSoSinsociety.Thecoursepresentsrecentdevelopmentsinframeworksforformulatingsystem-of-systemsproblems,lexiconfortheirarticulation,andanalysismethodologyfortheirstudy.Throughindividualandteamprojects,studentsgainexperienceinformulatingproblemsandapplyingtheoryandtechniques.Applicationsforteamprojectswillincludetransportation,spaceexploration,energy,defense,andinfrastructure,thoughothersarepossibleinconsultationwithinstructor.

UniversityofSouthernCalifornia

Systems/System-of-SystemsIntegrationandCommunication

Essentialsofsystemsandsystem-of-systemsintegrationfromtheperspectivesofbusiness,programs,andtechnology.Process,Legacy,andSystems-of-SystemsIntegration.VerificationandValidationmethods.Casestudies.


121

APPENDIXG:MISSIONENGINEERINGPROGRAMATOLDDOMINIONUNIVERSITY

Table15providesanoverviewof theonlymissionengineeringdegreeprogram identifiedbytheRT-171team.ThisuniquegraduatecertificateisofferedbyOldDominionUniversity(ODU)inMissionEngineeringandAnalysis(ODU,2017).

Table15:CoursesandDescriptionsofODU’sGraduateCertificateinMissionEngineeringandAnalysisCourseNumber CourseName CourseDescription

ENMA640 IntegratedSystemsEngineeringI(prerequisitetoENMA650)

Thiscourseexaminestheroleandnatureofsystemsengineering.Itisspecificallydesignedtoprovidethefundamentalunderstandingofsystemsengineeringandcomplexsystems.Thiscourseexaminesavarietyofsystemsengineeringtopicswithemphasisonthe:(1)developmentofthefundamentalsofsystemsengineering,(2)systemsengineeringlife-cyclemodelsandphases,(3)systemsdesignforoperationalfeasibility,and(4)anintroductiontoplanningforsystemsengineeringandmanagement.Thiscoursepreparesstudentstoassumetheroleofasystemsengineerinplanning,directing,conducting,andassessingsystemsengineeringinitiatives.

ENMA650 MissionAnalysisandEngineering(required)

Thecourseprovidesanoverviewofmissionengineeringandtheroleofmissionengineeringandthemissionengineeringovernmentacquisitions.Thecoursepresentsthetheoreticalfoundationsthatenableafullerrepresentationofcomplexproblemaswellastherequiredengineeringandmanagementapproachesneededtodealwiththehighlevelofcomplexityanduncertainty.Itappliesthetheoreticalfacetstospecificengineeringproblems/casesandexploresrobustapproachesgiventheconditionsoftheproblem.Developments,on-goingresearch,aswellasgapsinknowledgeandknow-howarediscussed.Prerequisites:ENMA640.

ENMA660 SystemArchitecture(elective)

Studentslearntheessentialaspectsofthesystemsarchitectureparadigmthroughdevelopmentandanalysisofmultiplearchitectureframeworksandenterpriseengineering.Emphasisisplacedonsystemsmodelingandenterpriseengineering.

ENMA702 RationalDecisionMaking(elective)

Thegoalofthiscourseistoenhancethestudent'sabilitytomakerationalandstrategicdecisionsincomplexsituations.Thecourseissplitintwomodules:decisiontheoryandgametheory.Thedecisiontheorymodulefocusesonhowindividualsmakecomplexdecisions,bothfromaprescriptive(ideal)anddescriptive(actual)perspective.Thegametheorymodulefocusesonstrategicdecision-makinginsituationswhereindividualsmustinteractwithoneanother.

ENMA715 SystemAnalysis(elective)

Thecourseisdesignedtoprovideanunderstandingoftheinterdisciplinaryaspectsofsystemsdevelopment,operation,andsupport.Thecoursefocusesontheapplicationofscientificandengineeringeffortstotransformanoperationalneedintoadefinedsystemconfigurationthroughtheinteractiveprocessofdesign,test,andevaluation.

ENMA750 SystemofSystemsEngineering(elective)

ComprehensivetreatmentofSystemofSystemsEngineering(SoSE),including;fundamentalsystemsprinciples,concepts,andgoverninglaws;complexandsimplesystems;underlyingparadigms,methodologiesandessentialmethodsforSoSEanalysis,design,andtransformation;complexsystemtransformation;currentstateofSoSE


122

CourseNumber CourseName CourseDescription

researchandapplicationchallenges.Explorestherangeoftechnological,human/social,organizational/managerial,policy,andpoliticaldimensionsoftheSoSEproblemdomain.

ENMA755 HumanSystemsEngineering(elective)

ThiscourseintroducestheconceptsofHumanSystemsEngineering,focusingondesigningsystemsthatincludehumancomponents.HumanSystemIntegrationandHumanFactorsEngineeringarediscussed,aswellasotherhumancentereddesignapproaches.Theroleofhumandatainsystemofsystemsdesignisexplored,andmethodstocaptureandrepresenthumandata,includingarchitectureframeworks,arepresented.Modelingandanalysisofhumancenteredsystemsisdonethroughhands-onprojects.

ENMA605 CapstoneCourse(required)

Inthiscourseyouwillapplyyourknowledgeofallfourcoursesthroughproject-basedlearning.Moststudentschooseaprojectwhoseresultswillbenefittheirownorganization.


123

APPENDIXH:ISO/IEC/IEEE152882015SYSTEMSANDSOFTWAREENGINEERING–SYSTEMLIFECYCLEPROCESSES

ThefollowingguidelinesaddresstheapplicationofSEtoSoS,basedontheDoDGuidetoSEforSoS (2008) and the International Organization for Standards/ International ElectrotechnicalCommission/InstituteofElectricalandElectronicsEngineers(ISO/IEC/IEEE)15288:

ISO/IEC/IEEE 15288: 2015 establishes a common framework of process descriptions fordescribing the life cycle of systems created by humans. It defines a set of processes andassociatedterminologyfromanengineeringviewpoint.Theseprocessescanbeappliedatanylevel inthehierarchyofasystem’sstructure.Selectedsetsoftheseprocessescanbeappliedthroughoutthelifecycleformanagingandperformingthestagesofasystem’slifecycle.Thisisaccomplishedthroughtheinvolvementofallstakeholders,withtheultimategoalofachievingcustomersatisfaction.ISO/IEC/IEEE 15288: 2015 also provides processes that support the definition, control andimprovement of the system life cycle processes used within an organization or a project.Organizationsandprojectscanusetheseprocesseswhenacquiringandsupplyingsystems.ISO/IEC/IEEE15288:2015concernsthosesystemsthatareman-madeandmaybeconfiguredwith one or more of the following system elements: hardware, software, data, humans,processes (e.g., processes for providing service to users), procedures (e.g., operatorinstructions),facilities,materialsandnaturallyoccurringentities.