Program Planning & Control (PP&C) Business … · Program Planning & Control (PP&C) Business Services Assessment Core Team Deep Dive Findings 2016 NASA Cost Symposium August 23-25,

Program Planning & Control (PP&C) Business Services Assessment

Core Team Deep Dive Findings

2016 NASA Cost Symposium August 23-25, 2016

Dorothy Rasco

Pre-Decisional: Internal Use Only Page 1

Pre-Decisional: Internal Use Only Page

Table of Contents

3 Introduction, Overview, Purpose, & Scope/Goal

9 Overview of Sources & Pre-BSA Data

14 Response to BSSC Questions

23 Back-up Slides & Discussion on Maintaining Data

2


Introduction, Overview, Purpose

& Scope/Goal

3


PP&C BSA Deep Dive Core Team

Dorothy Rasco JSC, Chief Financial

Officer

Rob Moreland HQ, Office of the Chief Engineer, Chair PP&C Agency Working Group

Debra Kolb JSC, Executive Assistant

Stephen Shinn GSFC, Chief Financial

Officer

Darwina Marks HQ, Human Exploration

& Operations Mission Directorate (HEOMD)

Kevin Rice JPL, Assistant CFO for Business Management

Amir Deylami HQ, Space Technology

Mission Directorate (STMD)

4


Business Service Assessment Context

Competition Model

Contract Assessment

Business Services

Assessment Agency Integration

Program/ Project

Planning

Tech Capability Assessment

NEW AGENCY OPERATING MODEL

IMPERATIVE: Establish a more efficient operating model that maintains critical capabilities AND meets current and future mission needs

5


PP&C BSA Team Major Milestones

Overall PP&C BSA Schedule:

ü  Core Team Kick-off F2F at HQ April 25 – 27

Develop Questionnaire April 25 – May 12

Distribute Questionnaire to SME’s/POC’s May 12

Core Team Review Questionnaire with SMEs/POCs May 13

Core Team Follow-up Review of Questionnaire with SMEs/POCs May 25

Receive Center/MD responses June 8

Core Team Review Center/MD Responses June 9 – 15

Core Team F2F at JSC June 16 – 17

Core Team Pre-briefs BSSC on Findings June 22

Core Team Briefs Draft Findings to SMEs/POCs June 28

Core Team Presents Observations/Findings to BSSC July 11-13

ü 

ü 

ü 

ü 

ü 

ü 

ü 

ü 

ü 

ü 

6


PP&C BSA POCs/SMEs

Center/MD POC PP&C SME PP&C Alternate SME

ARC Deborah Feng Thomas Paine N/A

AFRC Jack Gregory Jan Purifoy (Co-SME: Steve Sterk) Connie Hines

GRC Janet Watkins Mike Zernic Bob Sefcik

GSFC Nancy Abell Matt Ritsko Sherri Corbo

HQ Jay Henn Rob Carver N/A

JSC Mark Geyer Cathy Claunch Angela Bauer

KSC Kelvin Manning Bobbi Gnan Anna Henderson

LARC Cathy Mangum Yvonne Dellapenta Amy Radford

MSFC Robin Henderson Johnny Stephenson Craig McArthur

SSC Ken Human Jim Bevis N/A

ARMD Jon Montgomery Cathy Delaney N/A

HEOMD Kathy Nado Katie Gallagher-Boggs N/A

MSD N/A Lynn Irvine N/A

SMD Dan Woods Craig Tupper N/A

STMD Voleak Roeum Amir Deylami N/A

7


PP&C BSA Background & Scope/Goal

•  Background: The Budget and Management BSA core team presented key findings on the close integration of budget processes with project management practices. Based on these findings, the Business Services Steering Committee (BSSC) decided to collect some additional information and data on the (inter) related activities impacting budget processes, such as PP&C and work from the Council of Deputies (whose focus is on program/project management practices for NASA).

Scope/Goal: The PP&C BSA core team will collaborate and partner with the Council of Deputies team to evaluate the health of the Agency’s current PP&C functions, identify opportunities for optimization, and provide findings that will promote excellence, effectiveness, and efficiency across the Agency. This team’s focus will address the following areas of PP&C: Cost Estimating, Cost Assessment (including Joint Confidence Level), Resources Management, Scheduling, Earned Value Management, and PP&C Integration (Programmatic Assessment).

• 

8


Overview of Sources&

Pre-BSA Data

9


PP&C BSA Team Gathered and Assessed Data from a Wide Variety of Sources

• • • • 

• 

Deputies Team Report on A-SIP Actions, June 2016

NASA Deputies Council Program/Project Analysis Report Oct. 2015

PP&C community and Agency Working Group (e.g., PP&C Handbook (Rev. 1 draft)

Agency Questionnaire to assess health of PP&C; SME interviews, June 2016

Critical Issues Identified in GAO and NASA IG Reports -- Illustrative

Scheduling -- Require realistic cost and schedule estimates, well-defined and stable requirements, and mature technologies early in project development

Management -- Use metrics and controls throughout the life cycle to gauge when requisite level of knowledge has been attained and when to direct decision makers to consider criteria before advancing a program and making additional investments

EVM -- Consistently use EVM by projects as a tool for managing cost and schedule

Training and Knowledge Management -- Establish program/project management requirements for institutional KM and sharing

Cost Estimating -- Include information on cumulative prior budget authority and current cost ranges to enhance knowledge for funding decisions

NASA history on PP&C findings, internal and external, with sustained improvements, such as recently recognized in “NASA Assessments of Major Projects” (GAO-16-309SP, March, 2016)

“The cost and schedule performance of the NASA portfolio of major projects has improved over the past 5 years, and most current projects are adhering to their committed cost and schedule baselines”

10


• • • 

• • • • • 

PP&C BSA Data Overview

• • • 

• • • 

NASA’s PP&C Community is comprised of 2,114 professionals.

FTE/WYE Distribution FTE – 1,286 WYE – 828

FTE OPM Series 343 Series – 525 500 Series – 208 800 Series – 263 Other Series – 290

Survey results reflected 1,421 professionals as 343 & 500 OPM Series. •  OPM Series

•  343 Series – 736 500 Series – 685 • 

•  PP&C/Non-PP&C •  PP&C – 733

Non-PP&C – 687 • 

The PP&C BSA focused on six key functions which account for a total of 1,465 professionals.

Resources Management - 893 PP&C Integration - 239 Scheduling - 139

Cost Estimation - 91 Cost Assessment - 65 Earned Value Management - 37

11


Overall Team Impressions

Overall PP&C Team Assessment: –  Overall, the PP&C community has been addressing weaknesses and responding to external

stakeholders recommendations; Evidence of meeting our external commitments has improved. •  In recent years, the overall community has been very collaborative through various trainings, face-to-face

opportunities, and has made major progress in developing an Agency PP&C Handbook to aid in continuous improvement.

–  Concern there is limited advanced proficiency in many PP&C functions leaving a gap in the future with succession planning.

•  For example, Resources Management function has significantly lower % of FTE experts in comparison to other functions.

–  Concern Agency does not have adequate skills for PP&C Integration & Scheduling at all centers. •  PP&C Integration capability on programs/projects not as collaborative as in other areas and training is not as well

known, publicized, and/or attended.

Centers who have a Scheduling capability rely heavily on Contractor support. • 

–  It is unclear whether centers are right-sized across PP&C functions. •  Additionally, some smaller centers lack full range of PP&C capabilities.

–  No excess PP&C capability in the Agency; however, it appears there are significant OPM 343 series personnel doing non-business functions.

•  It is unclear whether the PP&C function has grown since 1993.

12

Conclusions


Past Future Decision Support

Decision Support

Program Controls

Program Controls

Reporting

Reporting Transaction Processing Transaction Processing

NASA’s PP&C capability appears to be adequately sized; however, its efforts should focus more on programmatic analysis, assessment, and decisional support.

13


Response to BSSC Questions

14


Overview: Follow-up Questions from the BSSC

•  Need more optimization opportunities: –  Please provide one or more major optimization opportunities to be more efficient in the area of resource

management/PP&C integration (could include necessary changes to existing process). Provide two additional major optimization opportunities to become more efficient within PP&C (Could include necessary changes to existing process).

– 

•  What within resources management do you consider transactional vs. analytical? Workforce: • 

–  Define any roles within PP&C that are inherently governmental and explain why? What is the optimal mix of FTE and WYE? Does the resource management area really require expert level since the PP&C integration area sustains a high expert level?

– – 

•  Rate geographic dependency by task (i.e. cost estimating, scheduling, JCL, etc.). –  On a scale of 1-5, 5 must be geographically co-located.

Provide rationale for rating. – •  How would you tailor PP&C and the sub-elements of PP&C activities to different

types of programs and projects (lifecycle, TRL, project type)? EVM Thresholds • 

15


Optimization Opportunity Areas

– – – 

– 

Note: Does not reflect SME inputs; Data not collected

Strengthening NASA’s PP&C Capability:

1. Consolidation – Offer agency enterprise service for Cost Estimating (including JCL) and EVM. 2. Scheduling – Leverage existing workforce to strengthen Scheduling capability (e.g. cross-training). 3. Collaboration – Improve PP&C Integration collaboration within programs/projects across Agency. 4. Training – Improve Agency PP&C training:

Increase training for Level 2 (Key Performer) and Level 3 (Expert) proficiency across all functions. Improve multi-discipline training across PP&C functions. Map course availability to proficiency gaps in training and provide a centralized warehouse for PP&C professionals to access course availability. Increase on-the-job (OJT) developmental opportunities for PP&C professionals.

5. Reporting – Eliminate requirement for integrated phasing plans for agency & centers. Reduce or eliminate property reporting requirements. Standardize reporting to better support various requests.

6. Shadow Organizations – Eliminate accounting/reconciliation/reporting for shadow organizations (matrix organizations, SID, etc.).

7. Sharing Programmatics – Program/project offices retain PP&C support at the lead center. Supporting centers only have technical responsibility, not PP&C, relieving center director from their oversight responsibilities.

8. Reviews – Reduce and/or combine agency/program reviews (e.g. SRB, BPR, DPMC/Full PMC, CMC, OIG, various audits, etc.). Review SRB requirements/frequency.

9. System Enhancements – Develop integrated budget formulation & execution system at WBS level. 10. Communication – Better planning and communication from the top-down to enable more efficient

organization (e.g. last minute data calls for routine actions, configuration control, etc.). 16


Resources Management Note: Does not reflect SME inputs; Data not collected

What within resource management do you consider transactional vs. analytical? Resources Management Role Transactional Analytical

Track and forecast based on obligation, cost, and workforce performance on a regular monthly basis.

Low performers = transactional High performers = analytical

Develop WBS and identify all necessary resources by element of cost (i.e., facilities and usage rates, FTE, travel, and procurements).

X

Establish the necessary funding profile to execute scope and schedule and allocate funding as required. X Establish the time-phased budgeted cost plan to achieve project execution in concert with scheduling, including PPBE submissions as required.

X

Capture and accumulate known risks and establish a process for allocating resources for unknown risks; trending and tracking unallocated future expenses (UFE).

X

Measure, analyze, and assess actual performance and impact on Estimate to Completion (ETC)/Estimate at Completion (EAC); update ETC/EAC as applicable.

5-10% (reporting) 90-95%

Generate and communicate monthly financial (costs and funds) performance for reporting including all contractors/suppliers.

Generate Communicate

Estimating cost when specialized cost estimating methodologies and datasets are not required.

Low performers = transactional High performers = analytical

Implement EVM as applicable (count EVM specialists under EVM function).

Data entry & calculations Analysis & communication

17


Function Current State Optimal State

Why? Inherently Govmtl? FTE WYE Total Inherently

Govmtl? FTE WYE Total

Total 968 497 1465

Cost Estimating Yes 58 33 91 Yes/

No = = = Modeling aspects of Cost Estimating does not need to be inherently governmental.

Cost Assessment Yes 43 23 65 Yes/

No = = = Non-decisional roles could be contracted.

Resources Management Yes 632 261 893 Yes/

No

Non-decisional roles could be contracted. Reduction over several years assumes less shadow org reporting and increased efficiency through attrition of low performers/transactional roles and increased training.

Scheduling No 18 122 139 No = Modest adjustment to reflect SME inputs. CS for schedule analysis; contractor to support transactional efforts.

Earned Value Management No 7 31 37 No = = = Right-sized to meet existing policy.

PP&C Integration Yes 211 28 239 Yes = CS for decision-making & strategy.

Workforce

– – – 

Note: Does not reflect SME inputs; Data not collected

Workforce: Define any roles within PP&C that are inherently governmental and explain why? What is the optimal mix of FTE and WYE? Does the resource management area really require expert level since the PP&C integration area sustains a high expert level?

18


Geographic Dependency

Can you rate geographic dependency by task (i.e. cost estimating, scheduling, JCL, etc.)? –  On a scale of 1-5, 5 must be geographically co-located; Provide rationale for rating.

Geographic Dependencies Function Regionalization

Opportunity1

(SME Response)

Regionalization Opportunity1

(Core Team)

Efficiency Gains2

(Core Team)

Cultural Change3

(Core Team) Rationale

Cost Estimating 3 2 4 Medium

JCL could be regionalized among centers or even HQ. Would eliminate the need to standup the practice at each Center; on-going estimating functions would likely need to be local, but periodic, milestone-driven estimates could be regionalized; modeling aspects would not require high geographic dependency.

Cost Assessment 4 3 2 Medium Complex or operational projects may be more efficient if personnel are

embedded within the program/project.

Benefits identified to having personnel embedded and geographically located within programs/projects to perform ad hoc analyses and conduct one on one interactions with personnel. Co-location enables stronger understanding of project content, performance, risks, and provides a significant advantage as projects implement changes.

Resources Management 5 4 1 High

Scheduling 5 4 3 Medium Schedule integration personnel would have higher geographic dependency; however, schedule creation and maintenance does not need to have high geographic dependency for low to medium complexity projects.

Earned Value Management 4 2 5 Low/

Medium

Some difficulty developing the capability at smaller Centers. Opportunity to leverage expertise from other Centers or Agency functions. A tailor-able agency-wide offering for smaller Centers would likely lead to more consistency and efficiencies at the agency level.

PP&C Integration 5 5 1 High Practitioners are often fully integrated with project teams, accepting

more management duties outside of traditional PP&C.

Legend: 1 Regionalization Opportunity (Scale: 1 to 5): 1 = Low Geographic Dependence, 5 = High Geographic Dependence 2 Efficiency Gains (Scale: 1 to 5): 1 = Low Efficiency Gains, 5 = High Efficiency Gains 3 Cultural Change to Regionalization (Scale: Low, Medium, High) 19


Tailoring Note: Does not reflect SME inputs; Data not collected.

How would you tailor PP&C and the sub-elements of PP&C activities to different types of programs and projects (lifecycle, TRL, project type)?

•  Cost Estimating: -  Increase thresholds that drive the types of cost estimates (grassroots vs. parametric).

Increase thresholds that require JCL. - •  Cost Assessment:

-  Eliminate and/or reduce agency and center oversight responsibilities for program and projects (e.g. Low TRL projects).

•  Resources Management: -  Develop an integrated budget formulation and execution system at WBS level.

Eliminate accounting/reconciliation/reporting for shadow organizations (Matrix orgs, SID, etc.). Eliminate requirement for integrated phasing plans for agency and center. Require centers to report WYE in SAP to automate Agency reporting/data calls (capability already exists).

- - - 

•  Scheduling: -  Eliminate and/or reduce agency and center oversight responsibilities for program and projects.

•  Earned Value Management: -  Increase thresholds that drive the types of EVM.

•  PP&C Integration: -  Reduce and/or combine program reviews (i.e. SRB, BPR, DPMC/PMC, CMC, OIG, audits, etc.).

20


EVM Follow-up Question Potential Threshold Changes

Thresholds ($M)

NASA Project LCC Contract Value

$0 - 20M Non-EVM Performance Mgmt Non-EVM Performance Mgmt

$20 - <$450M Non-EVM Performance Mgmt 32 Guidelines Compliance

$450M+ 32 Guidelines Compliance 32 Guidelines Validated

No EVM Required

•  Firm Fixed Price Contracts Service Contracts Small Cat 3 Missions Class D Missions In-House activities (up to $450M) Small Businesses (up to $450M) Universities (up to $450M)

• • • • • • 

“32 Guidelines” Requirements Compliance

•  IBRs EVM Reporting • 

• • • • 

Issue: Government will not conduct surveillance even though data issues can be reviewed with the Contractor.

Validated

IBRs EVM Reporting Validation Reviews Surveillance (on each contract & the EVM system)

21


– 

– 

– 

– • 

– – 

EVM Follow-up Question Potential Threshold Changes

1. Increase EVM project-level LCC Threshold from $150M to $450M Cost avoidance to relieve project-level (“in-house”) EVM.

2. Increase EVM Threshold for Validation on contracts from $100M to $450M Many contractors already have validated EVM system; Surveillance is where the savings exists Of the 18 contracts between $100M - $450M with EVM, saving a third of the EVM costs to avoid validation/surveillance is approximately $2.5M/year

Based on historical data, EVM cost is 0.5% - 1.0% of contract value Using very gross assumptions, and a list of contracts in the NASA portfolio, there is a potential savings of approximately $2.5M

3. Provide for EVM Threshold exemptions Exempt Class D/Cat 3 Small businesses/universities

Less aligned with other agencies on thresholds which could lead to negative perceptions by

external stakeholders (OMB, GAO, etc.).

Agency Thresholds for Contracts: Compliance Threshold

Validation Threshold

NASA ≥ $20M ≥ $450M DoD ≥ $20M ≥ $100M DHS ≥ $20M ≥ $50M DOE ≥ $20M ≥ $50M FAA < $10M ≥ $10M Note: EVMS Validation Reviews and Surveillance are required only for contracts ≥ the validation thresholds

22


Back-Up Slides

23


0

50

100

150

200

250

MSFC KSC

JSC

GSFC HQ

ARC GRC

LaRC

SSC

AFRC

HEOMD SMD

STMD

ARMD

Earned Value Management

Cost Assessment

Cost Estimation

Scheduling

PP&C Integration

Resources Management

PP & C Capability

1. Where does the business services capability exist today? –  Each field center has PP&C capability; however, it varies in depth and breadth.

Note: Graph is reflective of both FTE and WYE workforce.

24


0

50

100

150

200

250

MSFC KSC

JSC

GSFC HQ

ARC GRC

LaRC

SSC

AFRC

HEOMD SMD

STMD

ARMD

WYE

FTE

PP & C Capability


Above: FTE/WYE % Distribution

25


PP & C Capability


26

Documents

Program Planning & Control (PP&C) Business … · Program Planning & Control (PP&C) Business Services Assessment Core Team Deep Dive Findings 2016 NASA Cost Symposium August 23-25,