PS-24 Successful Path to DCMA EVMS Validation

IPMC 2012

Kevin Fisher

Orbital Sciences Corporation

fisher.kevin@orbital.com, 703-406-5084 , Kevin Fisher on LinkedIn

December 11, 20121

Learning Objectives

� Understand the Path that Orbital used to Achieve EVMS

Validation from DCMA the First Time Through

� Determine if Orbital’s Methodologies can Benefit your � Determine if Orbital’s Methodologies can Benefit your

Organization's Validation Project and/or Continuous

Improvements Efforts

2

TLO #1Agenda� Group Survey

� Understanding� Contract Requirements

� Self Assessment and Customers

� Building

� People

� System Tools & Architecture

� Processes & Documentation� Processes & Documentation

� Demonstrating

� Validation Review Preparation

� Validation Review Experience

� Decisions and Discussions

� Reflecting

� DCMA Observations

� Lessons Learned

� Questions

3

TLO #1Group Survey

� Who is Responsible for Performing EVMS in their job?

� Do you Submit Customer Deliverables (IMS and/or CPR)?

� Does your Contract Require EVMS Compliance?

� Does your Company have an Accepted EVMS?

4

� Is Anyone in a Company Preparing for a Validation Review?

� Has Anyone Been on a Team that Supported a Validation

Review?

� Did the Company Get Ready Before the Potential Award?

TLO #1Contract Requirement for EVMS

� DoD Earned Value Management Implementation Guide

5

TLO #1Self-Assessment and Customer Expectations

� 3 Operating

Groups

� LSG – 1998

Advance

Agreement

� SSG –

Commercial

6

Commercial

� APG – Mix of

Customers and

Requirements

� Dulles Campus

Heritage &

Culture

TLO #1PMI Conference (Nov08) – Best Practices

Joint NDIA/DCMA EVM Center Presentation

7

TLO #1DCMA Watch List from PAV

1. PM, CAM ownership, inattention to budgetary responsibilities2. Work authorization policies and procedures not always followed3. Budget/data reconciliation issues4. Lack of integrated management systems5. Baseline fluctuations and frequent replanning6. Current period and retroactive changes7. Improper use of Management Reserve and Undistributed Budget8. EV techniques not reflecting actual accomplishment9. Untimely, unrealistic Latest Revised Estimates10. Not monitoring progress in an integrated way

8

10. Not monitoring progress in an integrated way11. Zero budget work packages and control accounts12. Lack of vertical and horizontal traceability (critical path)13. Lack of critical subcontractor integration/oversight14. Material performance and accounting misalignment15. Not capturing and using cost and schedule data16. Poor root cause and predictive variance analysis 17. Not tracking corrective actions to resolution18. Managerial actions not demonstrated19. Lack of checks & balances (internal controls) and surveillance

TLO #1System Architecture

� APG System Architecture Alignment

9

TLO #1People - Team Success� Executives

� Program / Control Account Management

� Business Team

� Project Lead

� Program Control

� Planning / Scheduling

� Material Control

� Program Finance

� Business Systems / Cost Mgmt System Expert

� Contracts

10

� Contracts

� Technical Writing

� Supply Chain / Subcontract Mgmt

� Accounting Operations

� Compliance

� Consultants / 3rd Party Reviews

� Internal Surveillance & Audit

� DCMA – Contract Execution Surveillance

� Functional Management

� Prime Contractor / Customer Oversight

TLO #1People – Training & Support

� EVMS Training� Functional Department

� Humphrey’s Fundamentals Class

� CAM 15-Module Curriculum

� Supplemental / Special Topics Training Video-Based

� Mil-Std WBS Sample Program for Tools / Training

� New Program Kick-off Meeting Support

� One-on-One CAM / PMO Mentoring (External Support, Peer-to-Peer)

� Group Mentoring Sessions / Mock Interview Program

� CAM-Centric Tools

11

� CAM-Centric Tools� eCAM Notebook overlays Standard Directory Structure

� wInsight with Custom Views / Filters

� EVMS System Description Walk-Down and Storyboards

� CAM / PMO Interview Guide for Customer EVMS / Baseline Reviews

� CAM / PMO On-Line Assessments and Certification

� CAM Transition Process for Knowledge / Execution Transfer

� Professional Development� Planning & Scheduling Professional (PSP) - AACE

� Earned Value Professional (EVP) - AACE

� Project Management Professional (PMP) - PMI

TLO #1System Tools Architecture

� Integrated Cost and Schedule

� Single point of data entry

� Data moves between system pairs

� Commercial Products and Custom Solutions

PlanningCost

Management ReportingProposal

12

� Commercial Products and Custom Solutions

� Integrity Checks

� Any resource can produce incorrect data under the wrong circumstances

� Checks are scalable and continuously improving

� Consistency

� Our standard data flow is the nucleus for many of our processes.

� Similarity between program baseline, month-end processing, and baseline change processes.

TLO #1Processes & Documentation

� Industry Publications were Primary References as Documentation was Developed -

Numerous Documents Released by Government & Industry (DoD, NASA, NDIA….)

13

TLO #1Processes & Documentation

� Management Control System Description

� CDM-controlled & Submitted to DCMA / DCAA

� Supported by Progressively More Detailed Operational Docs� Process Guides

� Application Guides

� Desktop Instructions

� Business Operation Directives

� Templates and Instructional Forms

� Checklists

� Presentations Developed for Specific Events

� Training and Professional Development

� Integrates With Other Documents / Systems

Disclosure Statement

14

� Disclosure Statement

� Cost Estimating System

� Operating Group and Corporate Policies

� ISO Documentation and Processes

� Audited Results and Processes

� DCMA / DCAA

� Customer

� Internal Surveillance

� Corporate Internal Audit

� PwC

� ISO

� Industry Representation and Presentations

TLO #1The Validation Review –

Getting Ready and Data Drops� 3rd Party Assessment & Interviews

� Comprehensive Data Drop

� DCMA Used “Data Miner”

� Data Analysis Responses

� Cross Reference Checklist

� System Description Decomposed

� Mock Interviews for All Participants

15

TLO #1The Validation Review Experience

� DCMA In-Brief & Exit-Brief

� DCAA Operates Concurrent Review

� Company Executive Commitment Statement

� Executives may be Interviewed on Mgmt Reporting

� Company In-Briefs

� Program Brief

� System Overview

� Business Systems / Data Flow

� Data Trace / Storyboard

� Manager / Functional Pre-Completed

16

� Manager / Functional Pre-Completed

Interviews

� Program Manager

� Control Account Managers

� Business Manager

� Risk Manager

� Scheduling

� Material

� Subcontract Manager

� Accounting & Indirects

� VR Report / DRs / Contractor Response

TLO #1The Long & Winding Road To Acceptance

� Orbital Followed the 24-Step Process with DCMA / DCAA

� In Conjunction with External Surveillance on Compliant Programs

� 4-Years to Our Destination

� Jan 2008 Kickoff Using Orion Program

� Jun 2009 Progress Assist Visit / DCAA Initiates Separate Review

� Mar 2010 Validation Review Canceled due to Orion Termination

� Jul 2010 F6 Program Termination

� Dec 2010 VR Notification with Angels as Demonstration Program

17

� Dec 2010 VR Notification with Angels as Demonstration Program

�Angels 6-month Suspension due to Funding / 6-weeks to Resume

� Apr 2011 3rd Party Mock Review, DCAA Starts, DCMA Data Drop

� Jun 2011 Validation Review (2-weeks, 16 Personnel)

� Jul 2011 Initial Determination Report Received

� Aug 2011 Initial Determination Response Submitted with DR Actions

� Nov 2011 Discrepancy Report Actions Accepted

� Jan 2012 Letter-of-Acceptance / Advance Agreement

TLO #1New DCMA Process

� DCMA is Emphasizing

Contractor Preparation as

Shown in Compliance

Review Instruction

� Contractor Self-

Assessment using DCMA

18

Assessment using DCMA

Assessment Tools versus

Progress Assist Visit

(PAV)

� http://guidebook.dcma.

mil/39/index.cfm

TLO #1Significant Decisions & DCMA Discussion Topics

� Resource Loaded, Schedule-Driven EVMS Process

� Requires Enhanced Skill Set within Planning / Scheduling Group

� Schedule Margin Tasks

� ACWP at the WP-level / Charge Number Proliferation

� Subcontractor UB & MR Treatment Consistency

� Subcontractor BAC and PO Reconciliation by CA

19

� MR used to Budget for changes below Contract Threshold

� Quantifiable Backup Data (QBD) and X/Y Determination

� Communication Methods – Program Addendum, Program Directives &

Business Directives

� Indirect Rate Management, Publication, Thresholds & Variance Analysis

� Variance Analysis Reports

TLO #1DCMA Observations of Orbital Success

� Demonstrated Upper Management Commitment

� Attitude was Receptive & Cooperative Resulting in Effective

Relationship

� Quick Responses During VR & Initial Determination Led to

Accelerated Closure / Acceptance

20

Accelerated Closure / Acceptance

� Modern, Automated Tools with Limited Manual Processes

� High Level of Data Integrity

TLO #1Lessons Learned

� System Acceptance is a Big Investment

� Demonstration Program Customer will be Impacted

� Not a Part-Time Job – Staff Properly & Run like a Project

� Hit Your Deadlines & Make Sure you Pass the 1st Time

� Try to have Validation Applicable to All Contract Types (Development & Production)

� Consultants can be Important to Success but may give Conflicting Guidance

� Don’t let the Debates Continue – Pick a Path & Move Forward

� Prime Contractor Management

21

� Prime Contractor Management

� They are Responsible to Provide Oversight but Don’t Let Drive Decisions

� Supply Chain Flowdowns & Management

� Suppliers Require Significant Attention – Need Experienced, Knowledgeable Personnel on Team

� Sub Data can Undermine System Validation

� IBRs & Internal /External Surveillance Needs to be Ready out of the Gate

� Expect Personnel Turnover in CAM & BusOps

� Engineers not Generally Receptive

� Consider Program Control Staff to Bridge Knowledge & System Interface

� Prepare to Tailor Internal EVMS Reqs for Commercial / FFP programs

TLO #1Commit to Continuous Improvement

� Continuous

Improvement and

EVMS Maturation

are Important

Factors in

Performed

•Individual Program Analysis

•System Surveillance

•Surveillance Management

Repeatable

•EVMS Planning Across Programs

•Program Analysis

•System Surveillance

•Surveillance Management

•Enterprise (corporate/division) Application

•Program Analysis

22

Factors in

Improving

Efficiency,

Effectiveness and

Competitiveness

Defined

•Program Analysis

•System Surveillance

Quantitative

•Quantitative EVMS

•Enterprise Decision-making with EV

Optimized

•Continuous Process Management

•Innovation

Questions???

23