Successful Program Execution Successful Program Execution

Center for Systems and Software Engineering (CSSE)

Executive Workshop

15 March 2006

Center for Systems and Software Engineering (CSSE)

Executive Workshop

15 March 2006

Marilee J. WheatonGeneral Manager

Systems Engineering Division

Marilee J. WheatonGeneral Manager

Systems Engineering Division

©2006 by The Aerospace Corporation ©2006 by The Aerospace Corporation

2

•People and teamwork

•Controlling risk

•Processes

•Accountability

•Cost and schedule estimating

•Stability

•Industrial base

Key Attributes of Successful Program Execution

3

People and Teamwork

•Contractor execution team and PM should be first line of defense for mission success

– Second line is contractor functional organization check and balance

– Third line is government, FFRDC and SETA team

– Government is not staffed to be the first line of defense

•Space Quality Improvement Council (SQIC)

4

What Is The SQIC?

•Space Quality Improvement Council– An industry forum with government sponsorship– Identifies and addresses common problems and

solutions

•Executive-level engineering and quality control leadership from the NSS contractor community– Mission assurance processes– Product quality issues that affect all (Example: FPGAs)– Acquisition policy affecting quality

•Facilitated by The Aerospace Corporation– Carries forward consensus industry recommendations

to government sponsors– Not for attribution

5

Space Quality Improvement Council (SQIC)Key Accomplishments

•Contractor data sharing

– New SQIC/GIDEP NSS Advisory Forum, operational Sept. 2005

– Anomaly/mission failure investigations

• Standardized MOU developed and ready when needed

•Selective specs and standards reintroduction

– Industry representatives on all standards committees

– Industry consensus on specs and standards to put on contract

• SMC and NRO requiring that these be on new contracts

– Common playing field in bidding executable contracts

– Key element in assuring mission success

•Helps government be a “smarter buyer”

6

Smart Buyer 1

•Helping System Program Directors (SPDs) and acquisition executives become smarter NSS buyers

– Be aware of the demands and expectations placed on your industry counterparts

Smart Buyer 2

•Now that the buyer understands the contractor environment…

The buyer must formulate an executable plan and maintain an executable baseline.

7

How Do We Control Risk?

•Four variables in project management: cost, schedule, performance, and risk

•Need to define risk rigorously and cap it at an acceptable level

• If you cap the other three variables, risk grows continuously– Example: 1990s NASA

Mars missions, which failed

Assertion: Capping risk and driving schedule hard leads to best cost performance

8

How Do You Cap Risk?

•Use standards for engineering and manufacturing

– Appropriate test program, robust QA, PMP, etc.

•Rigorous system engineering and test

– Example: JPL Mars Rover “incompressible test list”

– Test like you fly, fly like you test

•Manage risk at a sufficiently senior level on a program

– Allowing lower levels to trade mission success for cost and schedule leads to unbounded risk

9

How Do You Cap Risk? (Cont.)

•Government/industry team manages risk incrementally

– Robust mission assurance tailored to risk assessment

– Contractor uses “buildup” process in design and test to define and manage risk

– Establish confidence at each major step—not just at the end of the program

– Find and fix defects early

•Establish environment that encourages problem reporting

– Example: Weekly Watchlist (“No Ambushes”)

10

Processes

•Use validated processes that ensure predictable, repeatable results– Processes represent lessons learned and best

practices from past experiences– Help prevent repeating mistakes

•Rigorously manage the baseline implementation•Fund and manage disciplined, validated technology

insertion•Require credible independent assessments prior to

program initiation and at key milestones

11

Processes (Cont.)

•Use standards and measure performance against those standards– Put core specs and standards on contract

– Examples: PM&P, SE, test– Government program office and FFRDCs ensure

that “best practices” are being used

•Independent flightworthiness and mission assurance certification

– Rigorous EELV mission assurance process has 2000 verification tasks prioritized against risk assessment

– Underpins “thumbs up” in Flight Readiness Review (FRR)

12

Accountability

•Nearly every failure in space systems resulted from a breakdown in accountability

•Need clear definition of accountability on government/contractor team– At every organization level and flows down to

individuals executing program

– Key program people remain in the job long enough to ensure they are accountable for their decisions that affect mission success

13

Accountability (Cont.)

•Key roles of government– Define requirements– Set risk level and assure responsible risk management– Ensure that proper practices are used– Incentivize contractor team and assess performance

•Contractor balances holistic enterprise perspective and tailored program needs

– Line organizations are accountable for the product

– Functional organizations are accountable for the processes

– Prime takes accountability for the quality of output by subcontractors and vendors

•Streamline government-industry decision-making process: In-plant representation

14

Cost and Schedule Estimating

•Recognizes that best value is not necessarily lowest cost bid

•Government must place value on non-deliverables essential to mission success (Examples: SE, MA, QA,…)

– Then industry will also value them

– Exclude “name-that-tune-in-three-notes” bids

•Has a well-established Independent Cost Estimating (ICE) and program control function

•Budget program to 80% confidence, including a management reserve sized by risk

– Expend reserves to execute unforeseen elements of baseline program—not new requirements

15

Stability

•Stable, manageable baselines—requirements, budget, and schedule

– Manage necessary but unplanned changes

– Rigorous systems engineering process for assessing impact of new requirements

– New requirements must come with new funding

•Allows trade spaces vs. “cast-in-concrete” requirements

– Capabilities, cost, and schedule

•Architectures that allow right-sized programs (can be executed in about 5 years)

– Regulates appetite of user community

16

Industrial Base

•Compete when in the best interests of the government– Government should recognize the need for stability,

longevity, and performance of industrial base

•Government aligns contract and fee structure to balance risk between government and contractor– Example: Titan IV contract

•A world-class supplier base– Multiple sources, particularly at sub-vendor level

•Government should require and enforce subcontract management plan– Put it on contract (How do we evaluate and enforce?)

17

Knowing When We Are Turning the Corner

•Effective programs that meet user demands with 100% mission success

•Efficient execution—meeting cost, schedule, and performance baselines

•Retirement of program risk as a function of time is measurable and visible

•Government and contractor teams meet their commitments

Attributes of World-Class Industry/Government Acquisition Team

Promises Made = Promises Kept