[American Institute of Aeronautics and Astronautics Space 2005 - Long Beach, California ()] Space 2005 - Policy Robustness Analysis of Space Exploration Architectures

American Institute of Aeronautics and Astronautics1

Policy Robustness Analysis of Space ExplorationArchitectures

G. R. Singleton* and A. L. Weigel†

Massachusetts Institute of Technology, Cambridge, MA, 02139

In this paper we develop a framework for evaluating the policy robustness of proposedNASA exploration architectures according to their vulnerability to changes in the policyenvironment. Several of NASA’s past programs have experienced failures when subjected toincreasing or changing demands from the policy environment, and there is a consequentrecognition that in order to be successful in their new initiative NASA must avoid a“business as usual” approach. In this paper we investigate historical policy issues affectingNASA programs, develop a typology to relate policy issues to system architectures, andpropose five guidelines to enhance the policy robustness of the exploration initiative. Finally,we developed metrics to assess the policy robustness of exploration architectures andevaluate the policy robustness of several proposed architectures. The policy robustnessanalysis framework developed in this work provides a means for assessing and improvingthe policy robustness of generalized system architectures.

I. Introductionn laying out his exploration vision President Bush explicitly identified sustainability as a critical element of a

successful exploration campaign.1 In doing this, the President is recognizing NASA’s past difficulties and callingfor efforts to prevent such problems with the new exploration effort. While NASA has accomplished incrediblefeats, it has also been host to programs that have had poor performance, cost overruns, and safety accidents. Ingeneral, all of these difficulties can be traced to specific technical causes, yet many if not all of them also havepreceding causes in the overall programmatic or management domain. In an effort to prevent such issues with thenew campaign we investigated policy robustness. Together with affordability, value delivery, and moderated riskexposure, they comprise the four pillars of sustainability.2 These four pillars of sustainability are necessary to givethe exploration campaign the capability to last over the next three decades or more, and are essential to avoidsuccumbing to many of the same problems that have befallen past programs. The exploration campaign still needs tohave a successful technical system and systems architecture, yet by requiring sustainability we are making the pointthat these technical characteristics alone will not ensure a successful system. The exploration campaign will need toendure over an extended period of time, and momentum solely from program initiation will be insufficient to keepthe project going. We maintain that the campaign should strive to embody the characteristics of sustainability inorder to maximize the likelihood of campaign success. Given the policy variability and the past susceptibility ofNASA projects to such factors, we contend that a sustainable exploration campaign must have policy robustness as acentral design requirement.

In this paper we explore the notion of policy robustness, meaning that the campaign should be capable ofadapting to shifting and modified government policies without failing to meet the needs of the critical stakeholders.3

As is illustrated in Figure 1 below, over the next 30 years we can expect fluctuations in the policy environmentalong some of the primary dimensions of change including, but not limited to, the political, financial, and worldconflict dimensions. The political dimension may change with each election. The financial dimension can fluctuatewith business cycles. And the geo-political dimension changes rapidly with levels of international conflict. Theexploration campaign is being designed and initiated within the context of a specific policy environment. Obviouslywe can not say which changes will occur, nor can we say when such changes might take place, but it would seemremiss to design a program which is unable to evolve beyond its initial policy environment. Even a cursory review

* Graduate Research Assistant, Technology and Policy Program, 77 Massachusetts Ave., Room E40-455† Assistant Professor, Department of Aeronautics and Astronautics and Engineering Systems Division, 77Massachusetts Ave., Room 33-404. Member AIAA.

I

Space 200530 August - 1 September 2005, Long Beach, California

AIAA 2005-6661

Copyright © 2005 by Greg Singleton and Annalisa Weigel. Published by the American Institute of Aeronautics and Astronautics, Inc., with permission.


of government experience with largeprograms would lead any reasonable observerto conclude that 1) changes are likely, and 2)programs, such as the proposed explorationinitiative, should plan for such changes.

Knowing that changes in the policyenvironment are inevitable, we initiated thisresearch in order to establish a framework forboth assessing and enhancing the policyrobustness of the NASA explorationcampaign. Our first task was to identify theprincipal stakeholders and their respectivepolicy levers for controlling NASA activities.We examined the historical use of thesepolicy levers, and developed a typology tocharacterize policy risks resulting from thesepolicy levers over the previous decade. Usingthese policy risks for guidance, we then

surveyed the literature to identify design paradigms that NASA could use to foster campaign success. We thendeveloped metrics to relate the identified policy robust strategies to several proposed exploration architectures, andevaluated the policy robustness of each architecture.

II. Policy Robustness PerspectiveA policy robust system is one that is capable of adapting to shifting policy demands while continuing to meet the

needs of the critical stakeholders. Whereas it might seem that this would be an obvious characteristic required of allsystems, in practice this is not the case. Many of NASA’s past programs, and indeed most government spaceactivities, were initially designed according to optimum vehicle mass, system cost, or technical system performance.Whereas these systems were free of technical design deficiencies, many of these programs encountered problemswhen they had to meet unexpected requirements from the policy domain. It is common practice when designinggovernment space systems to make critical design decisions as early as possible in the program lifecycle, since it hasbeen shown that changes to these critical decisions get increasingly expensive as the program proceeds.Unfortunately, these early program decisions also constrain the system’s ability to adapt to emerging policyenvironments without sacrifices to safety and performance.4 Policy robustness is more than a simple requirement tomake “correct” system design decisions from the outset; it is a requirement for a system design with the flexibility toadapt to changes during its lifecycle. Policy robustness involves an inherent recognition that in many cases auniversally correct decision can not be made due to changing conditions, and a more viable approach may be toaccommodate a range of alternatives.

Two important clarifications regarding policy robustness are that policy robustness is not the art of designing asystem to resist or counteract government political decisions, nor is policy robustness merely the codification ofpolitical maneuvering for program sustainment. From a technical systems perspective, politics represents the meansby which the government answers the yes/no question of whether a project is worth pursuing.5 Policy Robustness isnot an effort to design a system to actively resist this process, but does represent more generalized principles andoperating guidelines that aim to decrease the likelihood of running afoul of political decisions. We use the termpolicy robustness from the engineering perspective where a robust system is one that can function properly despitevariable conditions or various component failures. Accordingly, a policy robust exploration architecture would beone that could function within a variety of policy environments. We do not aim to design a system that can resistpolicy instructions, since national policymakers have proper authority over NASA’s actions, and any action toreduce that authority is inappropriate and also likely to encourage punitive behavior.6

Secondly, it is important for any large program to have a coalition of supporters to justify its existence,6 andprogram managers typically expend significant resources to ensure program sustainment by fostering this coalition.However, sustainment efforts are reactive real time efforts to ensure the continuity of an existing program, whereaspolicy robustness encourages a proactive system architecture that fosters inherent sustainability over the systemlifetime. Policy robustness requires a system to have the capability to deliver value to its stakeholders despitechanges in the national policy environment throughout its planned duration and beyond.

World Conflict Dimension

Skirmish

2005 2015 2020 2025 20302010 2035

Attack Regional Conflict

Financial Dimension

20352005 2015 2020 2025 20302010

Baby Boomers Retire Economic Recession Economic Boom

Presidential ElectionCongressional Election

2005 2015 2020 2025 2030 20352010

Political Dimension

Key Question: What set of guidelines will enable the architecture to berobust to policy changes over the life of the program?


Skirmish

2005 2015 2020 2025 20302010 2035


Financial Dimension

20352005 2015 2020 2025 20302010



2005 2015 2020 2025 2030 20352010

Political Dimension


Skirmish

2005 2015 2020 2025 20302010 2035


Financial Dimension

20352005 2015 2020 2025 20302010



2005 2015 2020 2025 2030 20352010

Political Dimension


Skirmish

2005 2015 2020 2025 20302010 2035



Skirmish

2005 2015 2020 2025 20302010 2035



Skirmish

2005 2015 2020 2025 20302010 2035

Attack Regional ConflictSkirmish

2005 2015 2020 2025 20302010 2035


Financial Dimension

20352005 2015 2020 2025 20302010


Financial Dimension

20352005 2015 2020 2025 20302010


Financial Dimension

20352005 2015 2020 2025 20302010


20352005 2015 2020 2025 20302010



2005 2015 2020 2025 2030 20352010

Political DimensionPresidential ElectionCongressional Election

2005 2015 2020 2025 2030 20352010

Political DimensionPresidential ElectionPresidential ElectionCongressional ElectionCongressional Election

2005 2015 2020 2025 2030 203520102005 2015 2020 2025 2030 203520102005 2015 2020 2025 2030 20352010

Political Dimension

Key Question: What set of guidelines will enable the architecture to berobust to policy changes over the life of the program?

Figure 0. Dimensions of Policy Variability


III. Prior WorkThis work was completed within the context

of a larger MIT and DRAPER effort on a NASAConcept Exploration and Refinement study toinvestigate exploration architectures. We beganwith a Value Object Process Model (VOPM)developed by our study team members, shownin Figure 2 at right, which characterized thestakeholders and value interactions for the entirecampaign.2 From this model we see that theCongress and the President are the twostakeholders with the most direct influence onNASA’s actions, and accordingly we focusedour analyses on these two primary policy relatedstakeholders. We then considered thestakeholder value lists from the VOPM andidentified those values which we felt coulddirectly affect the structure or design of theexploration architecture. Some values, such as“developing political capital”, are likely to beequally satisfied by all of the architectures;hence they are not useful as a discriminatorbetween exploration plans. On the other handthe executive branch need to foster dual usetechnology may be satisfied differentlydepending on the structure of the explorationarchitecture. We selected a subset of the VOPMstakeholder needs, shown circled in Figure 3,for our analyses based on whether they could beused to discriminate between proposedarchitectures. Starting with the stakeholders andvalues identified in the VOPM, we investigatedthe stakeholder policy levers and their affects onthe exploration campaign. Through our analysis,we developed policy robustness guidelines andmetrics for assessing the policy robustness ofproposed architectures. The final part of ourwork is the application of these metrics toproposed exploration architectures which weredeveloped by other members of the larger studyteam. We considered four architectures for Moon exploration, and seven architectures for Mars exploration.7

IV. Policy LeversAfter identifying the architecturally relevant stakeholder needs using the VOPM, we proceeded to investigate

their impact on NASA activities. We wanted to characterize the expression of stakeholder needs in interactionsbetween NASA and their policy stakeholders. Stakeholders use available policy levers to get results fromgovernment actors (NASA in this case), but these policy levers can be viewed as a risk for NASA since they requireappropriate action. To quantify this risk and any patterns of behavior, we performed a content analysis on historicalpolicy lever use by the President and Congress. The Congress’s authority over federal agencies is shared betweenthe authorization committees which construct authorizing legislation giving the agencies authority to conduct theiractivities, and the appropriations committees which annually appropriate and distribute money to the agencies. Inpractice most of the Congressional oversight is practiced within the appropriations committee, and their main policylever is the annual appropriations bill providing NASA funding. The actual bills are not very helpful for contentanalysis, but the conference report issued by the conference committee upon completion of the final legislation isfull of information including the findings, recommendations, and stipulations on the appropriated amounts from the

• Executive Branch Needs– Stewardship of Public Interests– Demonstrate US Leadership– Foster Industrial capacity & Dual Use

Technology

– Develop Political Capital– Sovereignty in Space

– Inspiration– Public Approval and Favorable Press

Coverage– Global Leadership & Foreign Policy

Capital– Progress on Initiatives– Execute Effective Exploration

Endeavor

• Congressional Needs– Stewardship of Public Interests

• Financial• Activities

• Benefits

– Demonstrate US Leadership

– Foster Industrial Capacity &Technology

– Demonstrate Support for SpaceExploration

– Develop Political Capital– Sovereignty in Space

– Inspiration

Figure 0. Architecturally Relevant Needs of Policy Stakeholders2

Explorationpolicy & regulations

funding

progress

awareness of benefits

experimentresults

experimentproposals

launch servicesproducts & services contracts

resource knowledge

space infrastructure& launch systems

technology &industrial capability

human capital

knowledge ofhuman and roboticexperience

exploration knowledge

developmental knowledge

operational knowledge

(mission objectiveshealth knowledge

safetyplan)

ScienceSciencegrants

science knowledgescience assets

Executive/CongressExecutive/Congress

foreign & nationalpolicy capitalawareness

Media

Commercial

Security

revenue & profitlaunch market

new busninesses

security assets

PublicPublicinspiration

prideAmericanidentityawareness

Educatorsexhibits

teaching resources

Internationalpartners

capabilities participation

products &services

jobs &stability

electionoutcomes

stewardship/constituency

events

securitybenefit

jobs &stability

knowledge

favorablepress

foreign policy capital

global leadership

studentsparticipation


revenue & profitpublicattention

eventsExploration

policy & regulations

funding

progress

awareness of benefits

experimentresults

experimentproposals

launch servicesproducts & services contracts

resource knowledge

space infrastructure& launch systems

technology &industrial capability

human capital

knowledge ofhuman and roboticexperience

exploration knowledge

developmental knowledge

operational knowledge

(mission objectiveshealth knowledge

safetyplan)

ScienceSciencegrants

science knowledgescience assets

Executive/CongressExecutive/Congress

foreign & nationalpolicy capitalawareness

Media

Commercial

Security

revenue & profitlaunch market

new busninesses

security assets

PublicPublicinspiration

prideAmericanidentityawareness

Educatorsexhibits

teaching resources

Internationalpartners

capabilities participation

products &services

jobs &stability

electionoutcomes

stewardship/constituency

events

securitybenefit

jobs &stability

knowledge

favorablepress

foreign policy capital

global leadership



revenue & profitpublicattention

events

Figure 0. Stakeholders & Value Flow2


conferees. We found that the conference reports from these annual NASA appropriations bills were a concise sourceof information on congressional views of NASA. NASA appropriations within the House are administered by theCommittee on Appropriations, Subcommittee on Science, State, Justice, and Commerce, and Related Agencies;whereas in the Senate they are overseen by the Committee on Appropriations, Sub-Committee on Commerce,Justice, and Science.8,9

In contrast to the appropriations bills of the Legislative branch, the Executive branch does not have a regularlyreleased record of its views on NASA. The president has many more direct policy levers such as politicalappointments, government objectives, or meetings to direct NASA’s activities. Unfortunately records of theseinteractions are unavailable for researchers. Fortunately we were able to gain some insight from the PresidentialDecision Directives (PDD) related to space issues.

V. Content Analysis

A. Congressional AppropriationsWe performed a content analysis on NASA appropriations legislation for fiscal years (FY) 1995 to 2005 by

categorizing segments of the legislation according to the type of operative clause.10-20 We found that segments of theconference report could be categorized according to the central goal or purpose of the segment (See example inTable 1).21 We called each of these discrete segments a single operative clause, and we wanted to see if we couldidentify any clear patterns relevant for policy robustness analysis. We examined these appropriations bills andanalyzed the conference reports in order to quantify their contents and characterize the type of instructions issued bycongress.

Table 1. Example Operative Clause CategorizationOperative Clause

CategoryLegislative Segment

Study

The conferees are aware of a study being conducted by the National Research Council … toaddress the station research program. If possible, the conferees would like the NRC to expandthat study to compare and evaluate the research programs of the ISS which can be accomplishedwith a crew of three and a crew of six; and, an assessment of the probable cost-benefit ratios ofthose programs, compared with earthbound research which could be funded in lieu of researchconducted on the ISS.17

ReportThe conferees share the concern of the House with regard to the establishment of a National

Program Office for air traffic management development and direct NASA to report to theCommittees on Appropriations by March 31, 2004 on efforts to establish the Office.19

Directive

The conferees remain strongly supportive of the Center of Excellence for Aerospace PropulsionParticulate Emissions Reduction established at the University of Missouri-Rolla’s Cloud andAerosol Sciences Lab and expect NASA to develop a plan to utilize the Center’s capabilities onan ongoing basis.19

Congressional Instruction GroupsWe grouped these operative clauses according to the type of topic discussed, and more detail is given in the list

below. A frequency distribution of these groups is shown in Figure 4:

1. Financial – Financial clauses are those that address and specify a specific level of funding forprograms in question

2. Study Clause – A study clause instructs NASA to commission a study to further consider an issue ofimportance to the Congress

3. Reporting Clause – A reporting clause instructs NASA to report back on the status of a specific endeavorafter a set interval or after a specific action has occurred.

4. Directives Clause – A directive clause instructs NASA to take a specific action which will change what it isthat NASA does

5. Goals Clause – A goals clause states the intention, desires, or goals of the congress, yet does notspecifically direct NASA to undertake any specific action

6. Review Clause – A review clause is a segment of the conference report where the conferees arereviewing or commenting on past NASA performance


Financial, 43.1%

Study, 3.8%Reporting, 16.2%

Directives, 26.6%

Goals, 3.3%

Review, 1.2%

Misc, 5.9%

Financial, 43.1%

Study, 3.8%Reporting, 16.2%

Directives, 26.6%

Goals, 3.3%

Review, 1.2%

Misc, 5.9%

Figure 0. Congressional Instruction Group Breakdown

7. Miscellaneous Clause – All other clauses not already counted, typically transfer authority, inspector general,and others.

Our initial content analysiscategorizations were a useful description ofwhat was contained within the conferencereports from the appropriations process. EachCongressional Instruction Group issubdivided to provide more detail on theeffect of the operative clauses. Within thedata, we found that NASA programs were justas likely to receive a funding increase, 12.2%,as a funding decrease, 11.5%. Anothertendency we noticed was for Congress todecrease the interval between their programprogress reports as they grew moredissatisfied with a program. Additionally,Congress frequently ordered NASA toperform studies in order to reconsiderdecisions that either Congress did not agreewith or that they felt needed moreinvestigation. In order to relate the contentanalysis to the exploration architectures,however, we realized that we would have tore-categorize the operative clauses accordingto their affects on the explorationarchitectures. We decided to group theoperative clauses according to which of thefour fundamental questions – who, what,why, and how – the clause addressed inregards to NASA activities.‡22 Within theappropriations reports we found that theclauses were limited to specifying WHATNASA should do, HOW NASA shouldconduct it, or HOW MUCH money NASAshould devote to the activity. By binning andtabulating these operative clauses we wereable to deduce the frequency of each type ofinstruction from Congress (see Figure 5), withinstructions related to HOW Much occurringroughly 40% of the time, instructions for Howoccurring 14% of the time, and instructionsfor WHAT should be done occurring 9% of the time. Another 30% of the instructions were irrelevant to theexploration architecture. We will discuss later how we used such information to formulate policy robust guidelines.

Architecturally Oriented Instructional Groups1. Why – Really the motivation for undertaking any government activity in the first place. Typically a

statement that expresses a need or scenario that needs to be addressed2. Who – These statements indicate which federal agencies have authority, control, or even standing to

participate in a specific domain. Other Agencies with operations in the space arena include the USAir Force, NOAA, USGS, DOD, and others.

3. What – Statements instructing NASA to execute or implement a specific program, activity, or objective.4. How – Guidelines and considerations that NASA should use when undertaking actions and program

‡ We omitted “when” and “where” from the typical list of six questions since we did not find instances whereCongress addressed them in architecturally significant ways.

Funding Actions(How Much), 43%Increases, decreases,upper limits, lowerlimits

Objectives Actions(What), 9%

Destination, duration,schedule, specificactivities

Execution Actions(How), 14%

Technology, int’lcooperation, commercialaspects, district location

ArchitecturallyNon-significant

Actions,31%

Reporting, etc.

Funding Actions(How Much), 43%Increases, decreases,upper limits, lowerlimits

Objectives Actions(What), 9%

Destination, duration,schedule, specificactivities

Execution Actions(How), 14%

Technology, int’lcooperation, commercialaspects, district location

ArchitecturallyNon-significant

Actions,31%

Reporting, etc.

Figure 0. Architecturally Oriented Instructional GroupBreakdown


strategies5. How Much – Funding decisions. The congress has final say on government agency spending levels, but the

president initially forwards his/her budget request to congress before they make adjustments.

B. Presidential Decision Directives (PDD)In contrast to annual Congressional oversight through the appropriations process, as the head of the executive

branch the President is involved more frequently, although less visibly, with NASA operations. Much of thepresidential leadership results from internal meetings and deliberations, but documents from these meetings areunavailable to outside researchers. However, the executive branch does issue periodic space policy guidance usingPresidential Decision Directives (PDD) that detail the presidential objectives, goals and guidelines. These policydirectives are infrequent and were historically issued once or twice per presidential administration beginning in1978.23 With fewer documents available to analyze, we did not feel that we could perform a meaningful contentanalysis, but we were able to draw some general conclusions about their contents.23-37 The first item in each of thedocuments establishes the agency responsible for the area of space policy being discussed. The documents thendescribe the goals or objectives of the agency, and may specify activities or programs that the agencies shouldexecute in pursuit of their objectives. Additionally the documents may specify guidelines or qualifications that theagencies should be mindful of while they pursue their overall goals.

VI. Policy Robustness StrategiesIn order to identify relevant strategies for policy robustness we researched essays and writings from space policy

researchers and other writings from the space policy domain. We investigated strategies that could be used toimprove NASA’s performance and accountability. We recognize that much of this commentary is critical afteraction analysis of what NASA should have done in the situations in question. We do not presume that the solutionare as simple as following the advice from the space policy literature, nor do we presume that the necessary actionswere easily knowable without the wisdom of hindsight. Rather, our objective was to select and synthesizerecommendations from this body of literature that could solve endemic and recurring problems. As opposed toproviding the solution that would have prevented the Mars Polar Lander loss, our focus was on identifying thestrategies that can be used to prevent the situation that allowed this loss to occur in the first place. Our goal was toidentify proactive actions that NASA can take today to improve their performance and survivability in an uncertainfuture. We are not describing prescriptive actions to address organizational errors; we are suggesting optionsavailable to improve the policy robustness of NASA’s exploration enterprise.

We selected five overall policy robustness strategies that could be implemented to increase the policy robustnessand hence sustainability of the exploration campaign. The five strategies are: (1) uniform funding profile, (2)frequent and sustained awareness and interest, (3) satisfaction of all stakeholders across the campaign, (4)anticipation and planning for policy risks of new technology, and (5) implementation of a flexible architecture. Weprovide a basic description, rationale, and implementation methods for each of the strategies below.

A. Uniform Funding ProfileDescription:

For a typical development project the funding profile (graph of annual spending vs. time, shown in Figure 6a)has a small initial design phase, a large development spike, and then a decrease to operational levels (nominal case).Our notion is that the exploration enterpriseshould maintain as flat of a funding profileas is possible, so that, as shown in Figure6b, the graph of annual spending over timehas little to no variation.Rationale:

NASA’s funding is appropriated underthe Science, State, Justice, and Commerce,and Related Agencies appropriations bill,and requests for large funding spikes aredifficult for congressional negotiators sinceincreases to NASA funding may necessitatedecreases to other entities under the sameappropriations bill. NASA’s appropriations

b) Flat Funding Profile

Time

$

a) Traditional Funding Profile

Time

$


Time

$


Time

$

Time

$

Time

$


Time

$


Time

$

Time

$

Figure 0. Funding Profiles a) Traditional vs. b) Flat


used to be managed under the VA/HUD appropriations bill, which meant that their funding requests were oftenbeing traded against spending for veterans or low-income housing. While NASA appropriations are now beingmanaged under a different appropriations bill, it is not clear that the agencies under this bill (State, Justice) will haveany less priority than those in VA/HUD. The difficult funding trades and political allegiances associated with allgovernment programs make it difficult for Congressional appropriators to do much beyond maintaining the fundingstatus quo.38 The following graph of NASA funding during the early 90’s illustrate the difficulties of gettingincreased funding levels. In Figure 7 we see the annual budget requests and forecast compared to the actual NASAappropriations over the same time period.39 It’s interesting that in each of the early years on the graph NASA’sprojections assume that they will receive significant budget increases in each of the out years. This trend continuesfor several years without any increases, and eventually the NASA projections are consistent with a flat fundingprofile. While additional funding may be needed at times, there exists a danger in relying on an increase sincecritical actions and decisions may be pushed back from the present on the assumption that more funding will beavailable in later years. When the funding fails to materialize, critical items may be further pushed into the futurewith dangerous results.

An additional funding profile concern is that projects with traditional profiles are oftentimes able to securefunding sufficient for the small pre-development phases, even if there is little hope of ever getting full developmentfunding.38 This might seem like an advantage for a program to at least get started, yet if the program’s budgetrequires funding beyond feasible levels then the program is bound to encounter problems. Such under fundedprograms typically limp along consistently failing to meet performance targets, doing serious harm to NASA’scredibility with appropriators in the process.

Implementation:Congressional appropriators will not provide excess program funding during any program phase, nor do they

allow prior year excess funds to be saved for later use. Hence it is not sufficient to simply request a constant fundinglevel; the entire program must be structured in a way that sufficient productive funding and development is requiredearly in the program lifecycle. Ideal early phase investments include risk reduction, technology development, andproof of concept activities that can be leveraged to improve the long-run project performance. Such spending can bethought of as insurance payments made early in the program to ensure that later efforts are more successful.

12

14

16

18

20

22

24

26

1990 1991 1992 1993 1994 1995 1996 1997 1998 1999Year

Do

llars

($B

FY

2000

)

Actual Budget1990 Request1991 Request1992 Request1993 Request1994 Request1995 RequestActual Budget*

12

14

16

18

20

22

24

26

1990 1991 1992 1993 1994 1995 1996 1997 1998 1999Year

Do

llars

($B

FY

2000

)

Actual Budget1990 Request1991 Request1992 Request1993 Request1994 Request1995 RequestActual Budget*

Figure 0. 1990's NASA Budgets, Actual vs. Requested39,*10-14


B. Frequent and Sustained Awareness and InterestDescription:

To the extent possible, NASA should strive to maintain stakeholder interest and awareness of their programsand activities.Rationale

In an ideal world, policy decision makers would recognize thatspace exploration is difficult and that the long lead times precludequick progress. Unfortunately experience has shown us that calls forpatience have had limited success in appropriations battles whencompared to more immediate spending priorities.6,40 NASA facessignificant competition for the attention and priorities of decisionmakers, and should therefore structure its programs to allow forfrequent milestones, early accomplishments, and emotional andpsychological investment of their key stakeholders (See Figure 8).Implementation:

To the extent possible NASA should structure the program anddevelopment plans to facilitate frequent concrete milestones and valuedelivery to stakeholders. There exist several concerns with stakeholderawareness, as the drive to get press coverage or meet schedulingconstraints was a significant factor in both the Challenger andColumbia accidents. There can be no bargains when it comes tohuman safety, but given the inevitability of such flight readinesspressures we propose that an architecture specifically designed toaccommodate the pressures will be safer and more resilient in the longrun when compared to a traditional architecture. Furthermore, not allmilestones need to involve human space flight, and unmanned probescan play a significant part in raising levels of awareness. Additionally,some milestones can represent the successful development and testingof capabilities within the exploration campaign. Whereas flightreadiness pressures contributed to several past NASA accidents, webelieve that structuring an architecture to meet such inevitabledemands enables a more functional architecture than the traditional paradigm.

C. Satisfy all stakeholders over whole campaignDescription:

Across the entire portfolio of activity in theexploration campaign, there should be at least oneaspect of the campaign that addresses a need ordesire of each of the important stakeholders.6

Rationale:Past programmatic activities have encountered

trouble by attempting to satisfy too manystakeholders at once. The approach we advocate isakin to a diversification investment strategy, andwe encourage NASA managers to view theexploration effort as a portfolio of independent butrelated programs that can come together to satisfythe necessary stakeholders (see figure 9). Such anapproach should simplify program and componentdesign requirements and enable more performancefrom the individual components. None of theindividual programs satisfy all of the stakeholdersalone, but the entire portfolio provides somesatisfaction to all of the stakeholders.

a) Traditional Milestone Profile

$

Time (yrs)1510 2050

Milestones

b) Phased Development & Early Milestones

B

$

Time (yrs)1510 2050

CA

Milestones

D


$

Time (yrs)1510 2050

Milestones


$

Time (yrs)1510 2050

$$

Time (yrs)1510 2050

Time (yrs)1510 2050 1510 2050

Milestones


B

$

Time (yrs)1510 2050

CA

Milestones

D


BB

$

Time (yrs)1510 2050

$$

Time (yrs)1510 2050

Time (yrs)1510 2050 1510 2050

CCAA

MilestonesMilestones

DD

Figure 0. Phased Milestones for ValueDelivery

Space Shuttle

ExecutiveCommercialDODScientists

Space Shuttle


LEO Booster

CEV

LandingModule

Geo EscapeBooster


LEO Booster

CEV

LandingModule

Geo EscapeBooster


Single Program Approach (Notional)

Portfolio Program Approach (Notional)

Figure 0. Distributed Stakeholder Satisfaction


Implementation:Independently managed and developed elements of the campaign should be assigned or targeted towards needs

of specific stakeholders, with the overall campaign managers ensuring that the needs of all of the criticalstakeholders are being met.

D. Anticipate and plan for policy risks of new technologyDescription:

Campaign managers should develop an architecture that is functionally mindful of the potential risks andconsequences of technology choices that involve politically sensitive technologies.Rationale:

Several of the technologies under consideration are immature and/or have the potential to encounter significantpolitical resistance. While we are unable to predict which technological investments will pan out or to what extent achoice will encounter resistance, we can be certain that there will be unexpected difficulties.6 Historically, thePresident and Congress have expressed opinions and specific guidance on launch vehicles and space nuclearpowered. Additionally, it is unclear whether in-situ resource utilization is permitted under current space treaties towhich the United States is a party.Implementation

As for the previous strategy, the most sensible approach to this strategy is also similar to an investment stylediversification strategy. To the extent possible the architectures should not be explicitly wed to technology choicesthat are risky, immature, or politically volatile. The architecture should be designed in such a manner that it has theflexibility to incorporate alternative technological choices (as depicted in Figure 10) without the need for extensiveredesign and rework.3

E. Flexible Architecture3,6,40

Description:NASA could seek to implement a flexible architecture that is capable of accommodating changes in funding,

goals, schedule, technology, partner involvement, and other elements of program execution (See Figure 11).

Rationale:The exploration initiative is a long term project for an agency to execute. While the government itself is stable,

its policies, agendas, and priorities are not. We challenge the reader to think of a long-term government project thatwas completed or implemented as it was originally envisioned. While it would be beneficial from an engineering

NotionalNotional

NotionalLaunch ConsumablesPropulsion

Heavy Lift

EELV

Nuclear Thermal

Solar electric

Bring With

ISRUNotionalNotional

NotionalLaunch ConsumablesPropulsion

Heavy Lift

EELV

Heavy LiftHeavy Lift

EELVEELV

Nuclear Thermal

Solar electric

Nuclear ThermalNuclear Thermal

Solar electricSolar electric

Bring With

ISRU

Bring WithBring With

ISRUISRU

Figure 0. Non Critically Dependent Architecture

Capability 1 Capability 3Capability 2Approach A

Approach D

Time

Approach I

Approach J

Approach S

Approach X

Approach Y


Approach D

Time

Approach I

Approach J

Approach S

Approach X

Approach Y


Approach D

Approach AApproach A

Approach DApproach D

TimeTime

Approach I

Approach J

Approach IApproach I

Approach JApproach J

Approach SApproach SApproach S

Approach X

Approach Y

Approach XApproach X

Approach YApproach Y

Figure 0. Technology Insertion Enabled Architecture


standpoint if program managers could negotiate or receive guarantees on stability from political decision makers, theharsh reality is that even if such promises were made they would most likely be broken. Due to the volatile nature ofgovernment policies, an architecture that is intentionally designed to accommodate change will be more policyrobust and more likely to survive than a static architecture requiring significant funding for change.Implementation:

Implementing an infinitely flexible architecture is obviously an impossible task, but we feel that a reasonablyflexible architecture is a feasible goal. The best way to implement such an architecture would be through intelligentmodularization of both functionality and physical modules. This modularization forces careful thought about thenecessary interface design and technical requirements early in the program’s development, and also allows formodification of components beyond interfaces with reduced rippling effects through the remaining systems. Suchmodularization could be achieved through the packetization of payload or spacecraft elements and through the useof a universal or nearly universal interface design. This packetization would allow various elements of thearchitecture to be swappable and would provide more interoperability between various elements while minimizingproblematic interference effects. The modularization of architecture elements would greatly enhance the flexibilityof the effort and allow it to accommodate a variety of changes from the policy domain.

F. Strategy SummaryIn the previous section we propose five strategies to enhance the policy robustness of the exploration architecture

based on the risk profile elucidated through our content analysis of stakeholder policy levers. These strategies areproposed as a means to reduce funding problems, maintain stakeholder interest, address diverse stakeholder needs,and provide flexibility with regards to technology and programmatic activities. The example shown in the priorsection cited the executive branch need to foster dual use technology, and discussed how such a need would becommunicated through a PDD specifying WHAT activities NASA should pursue. In order to comply with this needNASA could pursue the strategy to plan for new technology, and the suggestion to implement a flexible architecture.These policy robustness strategies would allow for development of new technologies at lower programmatic risk,while the flexibility reduces barriers to entry of new technologies. In the next section we propose metrics forevaluating and measuring the level of policy robustness of several architectures.

VII. Architecture MetricsIn the previous section we outlined and described the five principle policy robustness guidelines for the

exploration campaign. These are useful in providing direction for the exploration campaign, but provide little insighton assessing how well a given architecture fulfills the guidelines. Thus our remaining task was to devise methods tousefully relate these guidelines to early stage architecture design decisions. The metrics shown below in Table 2 arespecifically used for early stage architecture assessment, were designed to reflect the adherence of a givenexploration architecture to the policy robustness guidelines described in the previous section, and would most likelyevolve along with the program designs. The values of the metrics are relatively subjective, and are assessed incomparison to a baseline value. Table 2 contains four of the five policy robustness guidelines§, the relevantarchitecture characteristics, and a brief description of the metric definition. We conducted policy robustness analysisof proposed exploration architectures for both the Moon and Mars that are being considered by the Draper and MITstudy team using a standard rate and weight methodology. We do not discuss the architecture or ranking details inthis paper, but the authors are happy to provide them upon request.7,41 We performed this policy robustness analysisfor both the Moon and Mars exploration architectures.

When we analyzed the proposed Moon exploration architectures with our metrics we found that all of thearchitectures performed identically. We propose two main explanations for this lack of differentiation between thearchitectures. First, the architectures used in this study were initially selected and optimized based on the amount ofmass delivered to LEO. The architecture group used this metric as a proxy for overall mission cost. It turns out thatdue to this initial optimization the architectures share considerable similarities from a policy robustness standpoint.Secondly, many longtime observers of the policy process acknowledge that decision makers at the national levelrarely make programmatic decisions based on technical information.6 Decision makers at this level are concernedwith results and effects, not necessarily the details of how a plan is to be implemented. If we consider thearchitectures from this perspective, we see that they all deliver the same amount of mass (initial architectural

§ We did not derive any metrics for system flexibility since 1) to the extent that it could be measured it was alreadycaptured within the other metrics 2) due to the early stage of the architectures we did not feel that we could derivemeaningful metrics for architecture flexibility


assumption), have identical timelines, and to the current level of definition have identical funding requirements andrisk exposure. It is not unreasonable to conclude that at this point any of the key stakeholders would be effectivelyindifferent to whichever architecture is pursued. According to this analysis we find that our metrics effectivelyreflect the indifference of the key stakeholders on the issue of policy robustness between the architectures presented.

In contrast, we did seedifferentiation in policyrobustness rankings for the sevenMars exploration architectures,with four of the architecturesscoring identically and threescoring slightly lower. .Architecture differentiation wasseen in three of the policyrobustness metrics. One of thearchitectures scored slightly lowerthan the others since it had ashorter mission duration. Theshortened mission durationreduces the amount of science thatcan be completed, andconsequently the mission scoresworse on the “Adequate ScienceReturn” metric under the policyrobustness guideline to satisfy allstakeholders. A different MarsArchitecture scored worse on thetechnology risk factor due to itsreliance on nuclear technology. Athird Mars architecture had areduced rating for the technologyrisk score due to its reliance on In-situ resource utilization (ISRU), atechnology which is conceptuallysound although has yet to beproven and may have somepolitical and legal implications.

In prior sections of this paperwe provided an example of astakeholder need that ties throughto appropriate policy robustnessstrategies. The need for dual-usetechnologies was reflected in the flexible architecture and technology risk plan guidelines. In order to link thesepolicy robustness guidelines to the actual architectures, we developed three metrics that reflect the level of policyrisk inherent in any of the architectures from several specific policy-sensitive technologies. The policy risk wasevaluated based on a risk score regarding three potentially risky technologies: ISRU, heavy Lift launch, and nuclearcomponents. These metrics allow us to link the original executive branch need for dual use technology through tothe appropriate policy lever, to the mitigating policy robustness guideline, and finally to an evaluation of the actualarchitecture. This completes the link between stakeholder values and the system architecture design.

VIII. ConclusionThe basic philosophy of this investigation was that in the course of the exploration program NASA should strive

to take actions in the present that reduce the risk from policy uncertainties in the future. In this paper, we haveidentified the nature of the policy risks, first by conducting an exercise to identify the stakeholders and their corevalues, and then by pursuing the identification and quantification of policy levers by performing a content analysison the relevant policy related documents. From the NASA perspective we identified the consequences of policy

Policy Robustness Guideline

Architecture Characteristic

Architecture Metric

Uniformity of Funding Steady Yearly Program Funding

Annual Cost Uniformity Factor

Number of Unique Major Individual Elements in Architecture

Comparison to Benchmark

Risk of Cost Overruns Cost Overrun Factor Awareness and Interest

Number of Reportable Successful Events

Number of Planned Events per Year

Risk of Catastrophic Accidents (no loss of life)

Level of Programmatic Testing

Risk of Catastrophic Accidents (loss of life)

Degree of Crew Separation from Cargo

Satisfaction of all Stakeholders

Lowest Benefit of All Stakeholders

Lowest Benefit Level of All Stakeholders

Adequate Science Return

Percentage Scientific Payload Delivered versus Benchmark

Time Phasing of Benefits

Portion of Benefits Delivered Early in the Program

Risk of New Technology ISRU Risk ISRU Risk Factor Heavy Lift Risk HLL Risk Factor Nuclear Component Risk

Nuclear Risk Factor

Table 1. Policy Robustness Architecture Metrics


lever use as long term risks the agency must understand, and highlighted strategies that NASA could use as itundertakes the exploration initiative in order to enhance the policy robustness of the exploration architecture.Finally, we devised a series of metrics that can be used to both rank the level of policy robustness in a givenarchitecture and also to inform the further definition of the architectures in a policy robust manner. Our analysissuggests that NASA should seek to define its exploration plans with consideration not only to mass and cost but alsoto flexibility and adaptability. Such a policy robustness strategy requires more effort and expenditure in the shortrun, yet in the long run such efforts are necessary if the exploration initiative is to survive the inevitable policychallenges it will encounter.

AcknowledgmentsThe authors would like to thank the MIT/DRAPER CE&R study team – especially Eric Rebentisch, Paul

Wooster, Edmund Kong, Javier de Luis, and Ed Crawley – for their support and insights on this work.

References1The Vision for Space Exploration, National Aeronautics and Space Administration, February 2004,

<http://www.nasa.gov/pdf/55583main_vision_space_exploration2.pdf>2Rebentisch, E. S., Crawley, E. F., Loureiro, G., Dickman, J. Q., Catanzaro, S. N., "Using Stakeholder Value Analysis to

Build Exploration Sustainability," 1st Space Exploration Conference: Continuing the Voyage of Discovery, 20053Schulz, A. P., Fricke, E., "Incorporating Flexibility, Agility, Robustness, and Adaptability within the Design of Integrated

Systems - Key to Success?" Digital Avionics Systems Conference, 1999, p 1.A.2-1 4Hans, K. K., "System Development in the Federal Government: How Technology Influences Outcomes," Policy StudiesJournal, Vol. 28, 2000, pp. 313.

5Maier, M., Rechtin, E., The Art of Systems Architecting, CRC Press, Boca Raton; 2000, pp. 235, 2366Sadeh, E., Space Politics and Policy: An Evolutionary Perspective, edited by Sadeh, Eligar, Space Regulations Library; V.

2, Kluwer Academic Publishers, Dordrecht Boston; 2002, Chaps. 2, 3, 5, 7, 117Wooster, P., "Re: NASA Architecture Designs," Email to Author, January 18, 20058U.S. House of Representatives, Committee on Appropriations, Subcommittee on Science, State, Justice, and Commerce, and

Related Agencies, About the Committee: Jurisdiction, July 28, 2005,<http://appropriations.house.gov/index.cfm?FuseAction=AboutTheCommittee.Jurisdiction&SubcommitteeId=5>

9U.S. Senate Committee on Appropriations, Chairmen Cochran Announces New Structure for Senate AppropriationsCommittee, July 29, 2005, <http://appropriations.senate.gov/releases/record.cfm?id=232718>

10Conference Report on H.R. 4624, Departments of Veterans Affairs and Housing and Urban Development and IndependentAgencies Appropriations Act, 1995, Congressional Record of 103rd Congress (1993-1994), U.S. House of Representatives,Report: 103-715, August 26, 1994, Pages H9026-H9028.

11Conference Report to Accompany H.R. 3019, U.S. House of Representatives, Report: 104-537, April 25, 1996, Pages 538-539.

12Conference Report to Accompany H.R. 3666, U.S. House of Representatives, Report: 104-812, SEPTEMBER 20, 1996.,Pages 79-82.

13Conference Report to Accompany H.R. 2158, U.S. HOUSE OF REPRESENTATIVES, Report: 105-297, OCTOBER 6,1997, Pages 129-133.

14Conference Report to Accompany H.R. 4194, U.S. House of Representatives, Report: 105-769, OCTOBER 5, 1998, Pages291-297.



17Conference Report to Accompany H.R. 2620, U.S. House of Representatives, Report: 107-272, NOVEMBER 6, 2001,Pages 156-169.

18Conference Report to Accompany H.J. Res. 2, U.S. House of Representatives, Report: 108-10, FEBRUARY 13, 2003,Pages 1477-1487.



21Weigel, A. L., Hastings, D. E., "Measuring the Value of Designing for Uncertain Future Downward Budget Instabilities,"Journal of Spacecraft and Rockets, Vol. 41, 2004, pp. 111.

22Martin, M., "The Theoretical Foundations of Public Affairs and Political Strategy: Where Do We Go from Here?" Journalof Public Affairs, Vol. 1/2, 2002, pp. 330.

23National Space Policy, Presidential Directive / NSC-37, U.S. National Security Council, May 11, 197824National Space Policy, National Security Decision Directive / Number 42, Executive Office of the President, July 4, 1982


25Space Station, National Security Decision Directive / 5-83, Executive Office of the President, April 11, 198326Presidential Directive on National Space Policy, Executive Office of the President, February 11, 198827National Space Policy, National Space Policy Directives and Executive Charter / NSPD-1, Executive Office of the

President, November 2, 198928Commercial Space Launch Policy, National Space Policy Directive / NSPD-2, Executive Office of the President,

September 5, 199029U.S. Commercial Space Policy Guidelines, National Space Policy Directive / NSPD-3, Executive Office of the President,

February 11, 199130National Space Launch Strategy, National Space Policy Directive / NSPD-4, Executive Office of the President, July 10,

199131Landsat Remote Sensing Strategy, National Space Policy Directive / NSPD-5, Executive Office of the President, February

13, 199232Space Exploration Initiative, National Space Policy Directive / NSPD-6, Executive Office of the President, March 13, 199233Space-Based Global Change Observation, National Space Policy Directive / NSPD-7, Executive Office of the President,

May 28, 199234Convergence of U.S.-Polar-Orbiting Operational Environmental Satellite Systems, National Space Technology Council

Presidential Decision Directives / NSTC-2, Executive Office of the President, May 5, 199435Landsat Remote Sensing Strategy, National Space Technology Council Presidential Decision Directives / NSTC-3,

Executive Office of the President, May 5, 199436National Space Transportation Policy, National Space Technology Council Presidential Decision Directives / NSTC-4,

Executive Office of the President, August 5, 199437National Space Policy, National Science and Technology Council, The White House, September 19, 199638Byerly, R., Space Policy Reconsidered, edited by Radford Byerly, Jr., Westview Special Studies in Science, Technology,

and Public Policy, Westview Press, Boulder; 1989, Chaps. 1, 3, 439U.S. Congress, Reinventing NASA, Congressional Budget Office, 1994,40Lambright, W. H., Space Policy in the Twenty-First Century, edited by Lambright, W. Henry, New Series in NASA

History, Johns Hopkins University Press, Baltimore; 2003, Chaps. 6, 741Kong, E., "Re: Policy Robustness Ranking," Email to Author, February 25, 2005

Documents

[American Institute of Aeronautics and Astronautics Space 2005 - Long Beach, California ()] Space 2005 - Policy Robustness Analysis of Space Exploration Architectures