DCMA

EVMS STANDARD SURVEILLANCE OPERATING MANUALJANUARY 2008THE DCMA EVM CENTER DIRECTOR IS RESPONSIBLE FOR ISSUANCE AND MAINTENANCE OF THIS DOCUMENT

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008UPDATE STATUS

Date of Change Jan 08 25 Mar 08 10 Apr 08 20 May 08 11 Jul 08

Sections Amended and Reason for Change First Issue Clarifications to Sections 2.1 and 9 Updates to Risk Matrix form Change: 2.1 DFARS clause number updated to reflect Apr 08 DFARS change 1.1.5 EVM Center approval to EVM Center review. 9.4 Digital CAR Form link added 7.0 Clarified annual approval cycle and reaffirmed EVMS surveillance required by DFARS. SSP Template, Sec II, Step 1: from with an overall goal to review to with a requirement to review

i

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008TABLE OF CONTENTSFORWORD......................................................................................................................1 1.1 GENERAL OWNER INFORMATION........................................................................2

1.1.1 1.1.2 1.1.3 1.1.6 1.1.7

STANDARD SURVEILLANCE OPERATING MANUAL (SSOM)......................3 STANDARD SURVEILLANCE PROCESS FLOWCHART..................................3 STANDARD SURVEILLANCE PLAN (SSP).........................................................4 NATIONAL DEFENSE INDUSTRIAL ASSOC. INTENT GUIDE.......................5 THE EVMS CUSTOMER........................................................................................5

2.0 PURPOSE................................................................................................................5

2.2.1 CONTRACT MANAGEMENT OFFICE (CMO) ...................................................8 2.2.2 DCMA EVM CENTER ..........................................................................................10 2.2.3 PROGRAM MANAGEMENT OFFICE (PMO).....................................................113.0 SURVEILLANCE ASSESSMENT FRAMEWORK...................................................13 3.1 SURVEILLANCE DEFINITION................................................................................13 4.0 CREATE STANDARD SURVEILLANCE PLAN (SSP)............................................15 5.0 DEVELOP RISK-BASED APPROACH....................................................................16 6.0 DEVELOP SURVEILLANCE SCHEDULE...............................................................18 7.0 SSP APPROVAL PROCESS..................................................................................19 8.0 CONDUCT MONTHLY SURVEILLANCE................................................................20 9.0 CORRECTIVE ACTION REQUEST (CAR) PROCESS ..........................................21 9.1 ISSUING A CAR......................................................................................................23 9.2 CAR PURPOSE.......................................................................................................23 9.3 CAR SEVERITY LEVEL...........................................................................................23 9.4 CAR SUBMITTAL....................................................................................................26 9.5 DISPOSITION .........................................................................................................29 10.0 CAR CENTRAL REPOSITORY (AT PRESENT UNDER CONSTRUCTION) ........32 11.0 DOCUMENT AND REPORT RESULTS................................................................32 12.0 ESTABLISH AND MAINTAIN SURVEILLANCE FILES..........................................35 13.0 UPDATE STANDARD SURVEILLANCE PLAN AS NECESSARY.........................35 STANDARD SURVEILLANCE PLAN TEMPLATE ... 36

ii

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008LIST OF ATTACHMENTSATTACHMENT 1: SURVEILLANCE SELECTION RISK MATRIX ................... 47 ATTACHMENT 2: ANNUAL SYSTEM SURVEILLANCE SCHEDULE ..... 48

iii

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008FORWORD In its simplest form, Earned Value Management (EVM) is the discipline of managing projects successfully. It is the planning and controlling of authorized work to achieve cost, schedule, and technical performance objectives. Special emphasis is placed on efficiency and effectiveness in the execution of work through the development and operation of an Earned Value Management System (EVMS) to consider the application of people, systematic processes, and innovative tools and techniques. EVM helps project managers and their teams operate more effectively in the execution of risky - high dollar and complex programs. Successful project management requires well qualified and highly skilled project managers and integrated teams backed by management systems that provide immediate access to reliable and accurate data on project costs, schedule, and technical performance. The idea is that project managers and their teams perform best when they are well informed.

1

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008The loss of a Defense Contract Management Agency (DCMA) certification means the utility of a suppliers EVMS as a management process declines to the level where it does not serve its intended purpose. Concerns are raised regarding the suppliers ability to make informed decisions and to assess emerging cost, schedule, and performance issues in a timely manner. These concerns bring into question the accuracy and validity of performance measurement data that the government relies upon to manage its cost and incentive type contracts. These issues also raise concerns regarding the ability of the supplier to develop timely and accurate estimates-at-completion. The loss of a certified EVMS is considered a material internal control weakness due to the lack of compliance with applicable Department of Defense (DoD) policies and regulations as well as contractual terms and conditions. The suppliers decision to take a less disciplined approach to project management ultimately jeopardizes the long-term stability of the project and diminishes the purchasing power of the U.S. Government.

1.0 INTRODUCTION 1.1 GENERAL OWNER INFORMATION Routine surveillance is the best way to help ensure DoD gets the performance and dependability it expects from the Earned Value Management System (EVMS). This is the reason why the Standard Surveillance Operating Manual was developed. Basically, it outlines what oversight the EVMS will require and when. The focus of maintaining the EVMS while it is operating effectively will go a long way toward preventing major deficiencies and expenses later. If you have any difficulty implementing this standard or have any questions regarding the controls, operation, or maintenance of this document, you can seek help from the DCMA EVM Center.

2

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008Here are a few suggestions to help you get started performing standard and routine surveillance: Familiarize yourself with the surveillance process by going through this Operating Manual; Take a few minutes to review the steps of the Standard Surveillance Process Flowchart; and Consult the DCMA EVM Center for all your EVM needs.

1.1.1 STANDARD SURVEILLANCE OPERATING MANUAL (SSOM) The content in the Instruction and SSOM is the same; however, the format differs. By following the link to the SSOM, users benefit from a format that allows for numbered pages and a Table of Contents. This Instruction and SSOM both outline DCMAs standard surveillance process and provide detailed descriptions of the process steps to be used in developing a Standard Surveillance Plan (SSP). Whether accomplished jointly or independently, surveillance of Earned Value Management Systems should follow the same process steps in order to produce standard expectations and outcomes. As part of completing these steps, DCMA Contract Management Offices (CMOs), working through the DCMA EVM Center, must develop SSPs using a risk-based surveillance approach as described herein. How and when SSPs should be used and what should be included in them are described herein. For the purposes of interpreting this operating manual, wherever CMO is used, it should be interpreted as the person(s) responsible for conducting the surveillance and all surveillance process requirements. 1.1.2 STANDARD SURVEILLANCE PROCESS FLOWCHART The Standard Surveillance Process flowchart identifies the standard process that guides the surveillance team from one step to the next. Each process step contains sub-process and performance tasks. The more detailed process steps described in this document are highlighted by the off-page connector icon, a pink

3

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008home plate shape. The number inside the shape guides the reader to the place where that process step is explained. 1.1.3 STANDARD SURVEILLANCE PLAN (SSP) The SSP is an agreement among parties and a high level framework that sets expectations for surveillance. The SSP establishes the surveillance approach, risk criteria, and schedule. Surveillance may be conducted independently or jointly with team members participating from the supplier and government program office. However, the CMO is always responsible for identifying the contracts that require EVMS surveillance. The surveillance team is always responsible for performing surveillance regardless of supplier and/or program office participation. Surveillance requirements remain the same for either the independent or joint surveillance approaches, and are only considered compliant by following the standard surveillance processes. 1.1.4 STANDARD SURVEILLANCE PLAN TEMPLATE The SSP template serves as a pattern for the implementation of a standard surveillance process and the generation of standard outcomes. The SSP template includes a tailored risk evaluation determination that considers the different needs and emerging schedules on a contract level. 1.1.5 SURVEILLANCE REPORT Upon completing surveillance activities, it is the surveillance teams responsibility to produce written reports documenting surveillance findings. The report should include the assumptions, ground rules, and methodologies. The report should consider the view points of all surveillance team members. The draft report must be submitted to the EVM Center 10 days after completing the surveillance process. After EVM Center review, the surveillance team must make final report distribution within 20 business days of completing the surveillance process. Content, timeframe, and requirements for the documentation necessary to complete the report are covered later in this document. 4

STANDARD SURVEILLANCE OPERATING MANUAL JAN 20081.1.6 NATIONAL DEFENSE INDUSTRIAL ASSOC. INTENT GUIDE The National Defense Industrial Association (NDIA) EVMS Intent Guide contains the definitions and interpretations of the 32 ANSI/EIA-748 EVMS guidelines. In December 1996, these 32 guidelines were adopted by DoD as a new DoD 5000.2-R baseline criteria requirement. Therefore, the 32 guidelines contained in ANSI/EIA-748 are considered by DCMA as regulatory in nature, and will be used by DCMA to assess process conformance. 1.1.7 THE EVMS CUSTOMER DCMAs oversight of Earned Value Management Systems (EVMSs) is unique in that our customer for EVMS is a different and separate entity from the buying activity which in the past DCMA considered its sole customer. Because the Under Secretary of Defense for Acquisition, Technology, and Logistics (USD AT&L) is the entity that has the authority to give DCMA the DoD Executive Agent role and responsibilities for EVMS, it is this enterprise-level customer and not the buying activity that drives DCMA oversight activities. To this end, DCMA will use USD AT&L recognized Agency Level Performance Commitments (ALPCs) and related outcomes to measure our ability to positively influence supplier behavior in fielding robust and transparent management systems. It is essential that DCMA maintain a role of independence while maintaining a good working relationship with the buying activity as it executes its EVMS role and responsibilities. 2.0 PURPOSE Overseeing the implementation and use of the EVMS is only half the challenge. To ensure that the system continues to produce critical project information and remains compliant with the 32 ANSI/EIA-748 EVMS guidelines, a surveillance process must be put in place to assess the systems operation. The SSOM defines the roles and responsibilities for the surveillance of the EVMS. It defines the activities and interactions needed for successfully conducting surveillance 5

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008and what is expected of each participant in that process. This document also provides a process model for conducting surveillance in terms of when, and in what order, operational processes take place. To obtain reasonable assurance that the EVMS is being complied with and that performance outcomes are achieved; a common understanding of surveillance objectives, responsibilities, and assignments is essential. To this end, the surveillance team on a monthly basis should evaluate whether the internal controls governing the implementation of the EVMS are designed correctly and operate effectively. The surveillance team should obtain and document evidence from a number of sources, including the work performed by others recognized by the EVM Center. 2.1 POLICY Surveillance of the EVMS is mandatory for all contracts that require supplier EVMS compliance with the 32 ANSI/EIA-748 EVMS guidelines, regardless of whether a formal system validation is required. Surveillance begins at contract award, continues through validation (when required), and extends through the duration of the contract. Surveillance ensures that the supplier is meeting contractual terms and conditions and is in compliance with applicable policies and regulations. If changes are made to those terms and conditions, then a modification to the contract is required. Surveillance becomes mandatory through the inclusion of the Defense Federal Acquisition Regulation Supplement (DFARS) clause 252.234-7002. DoD uses DFARS in addition to FAR 52.234-4; a DFARS clause is substantially the same as the FAR clauses but tailors the requirement to DoDs unique needs. Current EVM DFARS clauses are: Solicitation Provision: 252.234-7001 Notice of Earned Value Management System Contract Clause: 252.234-7002 Earned Value Management System 6

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008 Legacy contracts with previous versions of the EVMS clauses

(252.242-7002, 252.234-7001), including the former CSSR clauses ( 252.242-7005, 252.234-7005), are all candidates for system surveillance based on risk. Federal Government Agencies outside of DOD use the FAR clause and Supplemental FAR clauses to implement EVMS Surveillance requirements (i.e. NASA FAR 1834.201) 2.2 RESPONSIBILITIES To avoid the duplication of efforts, minimize costs, and increase communication the CMO should make an effort to coordinate the government surveillance process with that of the supplier. A joint surveillance process between the CMO, the EVM Center, and supplier is encouraged and, if established, should be documented as part of the SSP. The SSP should be used to develop and update the yearly surveillance plan, whether surveillance is performed jointly or not. A number of stakeholders are involved in the surveillance of the EVMS however DCMA has primary responsibility for surveillance of the prime supplier and subtier suppliers with DFARS EVM requirements. Stakeholders recognized by DCMA include the supplier, program management office, and the Defense Contract Audit Agency (DCAA). All of these stakeholders may be surveillance team members. The supplier is not required to participate in the government surveillance process but is strongly encouraged to do so. DCMA encourages the participation and full cooperation of all stakeholders. Teaming is an essential ingredient to an effective surveillance program. Joint surveillance is when DCMA and supplier leads jointly conduct a review. Authority and independence, that is to say independent of the programs under review, are critical characteristics of this arrangement. The supplier team members must be independent of the management chain of the programs that it is responsible for surveying. Independence ensures that findings will be objective and that systemic issues on multiple programs will be identified. The surveillance team assigned responsibility

7

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008for implementing the surveillance process as described herein must also have sufficient authority to resolve surveillance process issues. To preserve the independence of results, the following guidance is provided for joint surveillance: a. Either surveillance lead (DCMA or supplier) may unilaterally recommend that a Corrective Action Request (CAR) be issued for noncompliant findings; b. DCMA, following its internal operating procedures, ultimately makes the final determination of non-conformance, severity, and applicability of a CAR(s); and c. Both surveillance leads (DCMA and supplier) must agree on the closure of a CAR(s). All non-compliant findings discovered at joint surveillance reviews are documented as CARs. All stakeholders with surveillance responsibilities are addressed in sections 2.2.1 through 2.2.5. 2.2.1 CONTRACT MANAGEMENT OFFICE (CMO) The CMO is responsible for the review of each contract to determine EVMS oversight requirements. The CMO is responsible for establishing a Memorandum of Agreement (MOA) with the PMO to include EVM Agency Level Performance Commitments (ALPC). The Product Division has an active role in ensuring the CMOs effectively implements ALPCs and related outcomes, and identifies appropriate activities (aligned to provisions of the MOA) to accomplish these outcomes in the areas relative to EVMS surveillance and data analysis. A MOA or the absence of one does not eliminate the CMO of its responsibilities related to the oversight of the EVMS. Working through the EVM Center, the prime CMO will also determine subcontract surveillance requirements and generate the supporting Letter of Delegation to be coordinated with other sub-tier CMOs.

8

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008For all contracts with EVMS oversight requirements, the CMO should reaffirm that the supplier has a DCMA-recognized EVMS as evidenced by an Advance Agreement (AA) or Letter of Acceptance (LOA). If a supplier does not possess a DCMA AA or LOA, the CMO should contact the EVM Center to request the appropriate compliance or validation review. The supplier EVMS is recognized (or accepted by DCMA) only after the Administrative Contracting Officer (ACO) has released the AA or LOA. At the corporate level, the AA or LOA should be administered by the Defense Corporate Executive (DCE). AAs that do not comply with the standard surveillance requirements contained herein must be updated. The CMO, working through the EVM Center, has the primary responsibility for EVMS surveillance in accordance with DFARS 242.302(a)(S-71) and the process steps as described herein. The following CMO personnel have surveillance responsibilities: The EVM Specialist is assigned overall responsibility for surveillance of the EVMS and as such, is the DCMA lead for surveillance team activities. This includes evaluation of supplier proposed alterations to the system, including changes to documented processes, procedures, and instructions. The EVM Specialist should work with the Program Support Team (PST), if one is established. The PST can assist in resolving and accomplishing system surveillance issues. The PST members include Systems Engineers, Industrial Specialists, Quality Assurance, etc. The PST members are responsible for accomplishing EVMS surveillance in their respective areas of expertise. The Program Integrator (PI) serves as the CMO focal point for overall program oversight. The ACO is designated as the agent of the government who is ultimately responsible for ensuring that the supplier complies with contractual terms and conditions.

9

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008The CMO will refer any specific questions or concerns regarding EVMS to the applicable EVM Center Product Lead for guidance. 2.2.2 DCMA EVM CENTER The DCMA Executive Director for Contract Operations & Customer Relations has been delegated the responsibility to oversee and enforce the Concept of Operations for EVM in conformity with this document. It is the EVM Center, working with the Product Divisions and CMOs, that is responsible for the implementation of DCMA policy and uniform procedures and processes to assess supplier ability to generate quality services and products in a timely manner. The EVM Center oversees and advises upon the implementation and sustainment of supplier EVMS and facilitate industry and Government maximization of EVM. It is essential for DCMA to operate in an integrated fashion in order to successfully execute its roles and responsibilities as the DoD Executive Agent for EVMS and to perform the most value added EVM-related services for our customers. The EVM Center has the following surveillance responsibilities: Responsible for ensuring that the CMO has adequate processes in place for assuring continued supplier EVMS compliance to ensure that EVMS data is accurate, reliable, and timely. This includes preparing a uniform standard surveillance plan template for CMOs to follow, to assess continuity and consistency in the operation of the EVMS and preparing and presenting step-by-step instructions for carrying out those plans (contained herein). Responsible for establishing Agency-level EVM performance commitments ensuring that CMOs have implemented meaningful system oversight measures with customers that tie to DCMA level performance commitments and identifying appropriate activities to accomplish these commitments.

10

STANDARD SURVEILLANCE OPERATING MANUAL JAN 20082.2.3 PROGRAM MANAGEMENT OFFICE (PMO) Key responsibilities for the PMO are to oversee the management of the program. The PMO sets program priorities and is responsible for defining the management and technology infrastructure including the creation of a governance structure that enforces use of standards and policies. The PMO establishes and maintains communications with senior management and the Program Executive Office, as well as providing direction and guidance to the program team with regard to the development and implementation of policies, methodologies, and reporting requirements. PMO responsibilities include providing effective project planning and control, decision support tools, and executive level reporting of schedule, cost, and performance measurement. In accordance with DoD regulations and USD (AT&L) Memorandum, April 23, 2007, Subject: DCMA EVM Roles and Responsibilities, the PMO is held accountable for complying with the DoD EVM reporting requirements. The PMO has the following surveillance requirements in accordance with the EVMIG and USD (AT&L) Memorandum, July 03, 2007, Subject: Use of Earned Value Management (EVM) in the Department of Defense: Working with DCMA, establishment of a Memorandum of Agreement (MOA). Keeping DCMA informed of actions and matters that could affect system surveillance. Assisting in the resolution of problems cited in surveillance reports by providing required support to DCMA. Reviewing, evaluating, and analyzing performance reports and schedules and bringing issues to the attention of DCMA. Apprising DCMA of the adequacy and usefulness of surveillance reports and, where necessary, stating required changes to reporting practices.

11

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008 Participating as members of the surveillance team (at the PMOs discretion). 2.2.4 DEFENSE CONTRACT AUDIT AGENCY (DCAA) The Defense Contract Audit Agency, under the authority, direction, and control of the Under Secretary of Defense (Comptroller), is responsible for performing all contract audits for the Department of Defense, and providing accounting and financial advisory services regarding contracts and subcontracts to all DoD Components responsible for procurement and contract administration. Because of the cost reporting requirements surrounding a compliant EVMS the accounting and financial related guidelines hold a key importance in the successful development and capabilities demonstration of the system. According to the DCAA Audit Program, Activity Code 17750, Part C-1, the accounting and financial aspects of 14 of the 32 guidelines required for a compliant system fall under the purview of DCAA. At DCMAs request, and under DCMA lead, the DCAA may support any or all of these in surveillance activities, as appropriate. When surveying accounting consideration guidelines, it is helpful to solicit DCAAs help wherever practical. Therefore, close coordination between DCMA and DCAA is required in the preparation of the surveillance plan to make a determination of the effective implementation of the system. The DCAA has the following surveillance responsibilities: Reviewing the supplier accounting system for compliance with Disclosure Statements and contract provisions, including verification of actual costs. Determining the accuracy and reliability of the financial data contained in the contract cost reports. 2.2.5 SUPPLIER The supplier is responsible for developing and implementing an EVMS compliant with the 32 ANSI/EIA-748 EVMS guidelines. The supplier is also responsible for

12

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008ensuring that its EVMS is implemented on a consistent basis, that it is used effectively on all applicable government contracts, and that EVMS clauses are flowed down to subcontractors when required. This responsibility is independent of DCMAs responsibility to develop and implement a surveillance process. 3.0 SURVEILLANCE ASSESSMENT FRAMEWORK Each CMO is required to have a formal surveillance plan as part of the Concept of Operations for Earned Value Management, August 2006. The decision to perform surveillance either independently or jointly is a decision that needs to be made between the CMO and the EVM Center. Not having a surveillance strategy or agreement in place with the supplier shall not prevent the CMO from accomplishing routine system surveillance as outlined in this plan. The purpose of the SSP is to establish the acceptable requirements for system surveillance. Because each supplier and system differs in surveillance needs, it is the responsibility of the surveillance team, working through the EVM Center, to tailor the SSP to match standard surveillance requirements to the particulars of the contract. 3.1 SURVEILLANCE DEFINITION EVMS surveillance consists of essentially two parts: Effective supplier implementation and maintenance of documented processes, procedures, instructions, use of tools in the EVMS process and techniques over time; and Surveillance results are documented and communicated to all stakeholders in a timely manner.

13

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008

START SURVEILLANCE

DCMA EVMS VALIDATION OR TRI SERVICE ACCEPTANCE NO

DECISION PROCESS FOR EVMS ACCEPTANCE

YES

1SSP CREATE STANDARD SURVEILLANCE PLAN

DEVELOP RISK BASED APPROACH

2

RISK CRITERIA

SURVEILLANCE SCHEDULE

DEVELOP 3 SURVEILLANCE SCHEDULE SSP

4NO APPROVED BY DCMA EVM CENTER COMPLETED SSP YES

RISK CRITERIA SURVEILLANCE SCHEDULE

5CONDUCT SURVEILLANCE

CAR REPOSITORY

CORRECTIVE ACTION REQUEST (CAR) PROCESS

6

7YES

ARE CORRECTIVE ACTION REQUESTS NECESSARY ?

NO

DOCUMENT AND REPORT RESULTS

ESTABLISH AND MAINTAIN SURVEILLANCE FILES

UPDATE SSP AS NECESSARY

CONTINUE SURVEILLANCE

FIGURE_1. STANDARD SURVEILLANCE PROCESS

14

STANDARD SURVEILLANCE OPERATING MANUAL JAN 20084.0 CREATE STANDARD SURVEILLANCE PLAN (SSP)1

Per DFARS 242.302(a)(S-71) DCMA has the responsibility for surveillance of the suppliers EVMS, to ensure ongoing compliance with the 32 ANSI/EIA-748 EVMS guidelines. EVMS surveillance begins at contract award, continues through the compliance or validation process, and extends throughout the duration of each contract.

4.1 CONSTRUCTION OF THE SSP The SSP uses a risk-based determination to generate risk ratings for each of the nine processes. The scope and timeframe of the reviews are discussed in the surveillance scope and schedule section of the SSP. When developing the SSP, the template (later in this document) is followed explicitly. Using the template (whether jointly or individually) ensures that surveillance is being performed in a uniform and consistent manner and produces repeatable results. The SSP template also ensures that all aspects of surveillance are defined up front and understood for each supplier. The SSP template includes definitions regarding the scope of each review, schedule, expectations, inputs, results, and follow-on duties. If surveillance is to be conducted jointly, the SSP template is used and identifies the joint team participants by role and name. 4.2 DEFINITION OF RISK-BASED APPROACH EVMS surveillance following a risk-based approach is performed by the surveillance team on a continuing basis where actual and perceived risks have been correlated to management processes and guidelines. The key processes include organization, scheduling, work and budget authorization, accounting, indirect management, managerial analysis, change incorporation, material management, and subcontract management. Risk-based assessments should be carried out for management processes and guidelines on a yearly basis.

15

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008The idea behind a risk-based approach is that the surveillance team focuses on the higher risk processes while reducing focus in the lower risk areas. Risk-based surveillance translates to increased time spent on processes and guidelines that have the greatest risk of unfavorably affecting system integrity. After the riskbased surveillance plan is worked through the EVM Center, it is then shared with all other stakeholders. 5.0 DEVELOP RISK-BASED APPROACH2

The SSP is used as a framework for each supplier surveilled by the surveillance team. Key process and guideline risk is determined by the data and information gathered from the supplier EVMS. A higher risk rating equates to more frequent surveillance activity and typically requires a more intense review of the processes. (Note: Processes and applicable guidelines are defined in the Earned Value Management Implementation Guide (EVMIG), Figure 2-1). A more intense review is defined as occurring with greater frequency, using a larger team, and reviewing data and information in greater depth for more contracts.

5.1 DEVELOP RISK SURVEILLANCE CRITERIA The surveillance team will identify and select programs/contracts for surveillance based on the risk associated with each program/contract using an algorithm that assigns relative weights and scales to each risk area. Risk factors include: program phase; earned value management experience; total contract value; value of prime and critical subcontract work remaining; value of material remaining; value of management reserve; number of baseline resets; cost, schedule, and at completion variance percentages; critical path float; baseline volatility; indirect costs volatility; past surveillance results; and time since last review. Once the programs/contracts have been rated and ranked, the surveillance team, working through the EVM Center, will generate process and guideline risk ratings that aid in identifying system risk. When determining risk the surveillance team 16

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008should be as objective as possible. Although several characteristics in rating process and guideline risk may allow for a subjective call, the surveillance team is expected to have sound reasoning for determining the risk level for processes and guidelines. The surveillance team must populate the surveillance schedule to reflect the high and medium risk areas that cover high impact contracts. Those areas that are determined to be low risk meaning there is a low probability that a key process deficiency will adversely affect the timeliness and accuracy of data, may be reviewed less frequently using fewer resources. When the risk-based assessment determines processes are low risk, concrete evidence must be provided by the surveillance team as to why those processes and associated guidelines retain a low risk status over time. Objective evidence can be gleaned from ongoing surveillance reports showing reductions in errors, data integrity improvement, implementation of corrective actions showing improvements, and any other favorable evidence proving effective project management. Care must be taken to document and establish a baseline reference point from which future measurements will correlate back to in order to generate valid assessments. Attachment 1 Surveillance Selection Risk Matrix shows a contract in algorithmic form that is currently in the development phase with a contract budget base of well over $100M being managed by a program manager with 5 1/2 years of EVM experience. The prime supplier and multiple sub tier suppliers are responsible for 40% and 60% of the value of remaining budget respectively with more than 30% of remaining budget associated with material (non-labor) work. Management reserve makes up 10% of the remaining budget while the program/contract has been rebaselined once in the previous year and reports a cumulative -15% cost variance and -10% schedule variance. The calculated critical path to contract completion is -15 work days float with an average of a 5% change to the baseline over the last 6 months and an indirect cost increase of 7% since contract

17

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008inception. The previous joint surveillance review was conducted over a year ago which identified significant system deficiencies that continue to require resolution. For each Risk factor, multiply Weight amount by High (3.00), Medium (2.00), or Low (1.00) and list in Score column. Then total the Score column. If Total Score is between 3.00 and 2.5, then program is rated High Risk. If Total Score is between 2.5 and 1.5, then program is rated Medium Risk. If Total Score is below 1.5, then program risk is Low. According to this algorithm, the risk score for the program/contract is 2.6 out of a possible 3.0. Using the risk algorithm to determine the score for all other programs/contracts allows each to be rank-ordered to identify which should be reviewed more frequently. One or more processes are listed under each high, medium, and low risk criteria on Attachment 1. The processes serve as the program/contract selection criteria when developing the system surveillance schedule.

6.0 DEVELOP SURVEILLANCE SCHEDULE Once it has been determined how the surveillance will be conducted following a3

risk-based selection approach, a surveillance schedule is developed by the surveillance team working through the EVM Center. The standard surveillance process as described herein includes criteria for identifying each process and guidelines at risk, the approach for selecting contracts and the frequency, intensity, and schedule of reviews. Interviews with the supplier PM, CAM(s), and other key team members are an essential part of ensuring continued guideline compliance. The surveillance team should continuously verify that supplier management personnel are using the EVMS to identify problems, develop solutions, and implement corrective actions where necessary. The surveillance team is required to perform system surveillance on a monthly basis and assess all 9 processes and 32 ANSI/EIA-748 EVMS guidelines annually unless a waiver 18

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008is granted by the EVM Center. The intensity of the review depends on the risk level; the higher the risk, the more intense the review. For surveillance teams with multiple programs/contracts, review the high and medium risk processes for each program/contract as determined by the risk selection criteria from Attachment 1. If a process is considered low risk, then objective evidence by the surveillance team must be provided to the EVM Center confirming the low risk status for each guideline based on a review of one or more programs/contracts. The length of time needed to conduct a surveillance review varies depending on many factors. The number of days, work hours, and resource mix vary. Individual site surveillance circumstances are to be discussed by the surveillance team with the EVM Center before agreement is reached on the proper length of time for each monthly review, given known constraints. The surveillance schedule is then adjusted to fit the needs of each annual surveillance assessment. It is important to collect all the data and reports needed to perform surveillance and to request that the right information be available when conducting surveillance. In preparation for reviews, some data may be required ahead of time. The schedule will reflect the process(es) and guidelines to be examined each month, the programs/contracts involved, and the frequency of the reports. The frequency of these reports is determined by logical grouping of processes, programs/contracts, by considering the intensity of review, etc. See Attachment 2 for the annual EVMS surveillance schedule format. 7.0 SSP APPROVAL PROCESS The approval process for release of the SSP (including each risk matrix and the4

annual EVMS surveillance schedule) is standard for all programs/contracts and CMO locations. The CMO must coordinate each initial calendar-year schedule (January December) and updated matrices with the EVM Center for approval 19

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008prior to commencement of the new year. As stated previously, DFARS requires system surveillance. Therefore, absence of an approved plan does not relieve the CMO from system surveillance responsibilities. 8.0 CONDUCT MONTHLY SURVEILLANCE How surveillance is performed, who performs it, who is part of the team, what5

tools are used, what documentation and data are required, as well as other issues, are further defined in the SSP. Although joint surveillance with the supplier and PMO is desirable, it is not required in order for the CMO to perform its EVMS surveillance responsibilities. While the supplier is ultimately responsible for the proper implementation of the EVMS, the CMO and EVM Center acting on behalf of DoD as the Executive Agent for EVMS are responsible for verifying ongoing supplier compliance. The absence of an AA or LOA does not relieve DCMA of performing EVMS surveillance on government contracts for which EVM is required. Similarly, no formal delegation from a DoD program office is required for DCMA to perform EVMS surveillance on suppliers for which EVM is required. For non DoD government agencies, such as NASA, DCMA needs an MOA stating that EVMS system surveillance will be performed and addresses reimbursement for the effort. If resource unavailability impacts efforts to conduct EVMS surveillance, the CMO is required to notify the EVM Center immediately. As stated previously, the surveillance team is required to perform system surveillance on a monthly basis and assess all 9 processes and 32 ANSI/EIA-748 EVMS guidelines annually unless a waiver is granted by the EVM Center. Additionally, the surveillance team must exercise due professional care in the surveillance of the supplier system. One important tenet of due professional care requires applying healthy skepticism. This includes a critical assessment of the tools, procedures and processes, and how they are used to manage the work. Healthy skepticism must be applied with the same considerations when 20

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008evaluating outputs from the EVMS as exemplified through discussions with the PM, CAM, and other team members including project planning and controls staff. Discussion and findings are to be documented using the DCMA CAR process. 9.0 CORRECTIVE ACTION REQUEST (CAR) PROCESS If deficiencies are found during the course of the surveillance process it is the6

surveillance teams responsibility, working through the EVM Center, to issue a written Corrective Action Request (CAR). All EVM, EVMS, or other Earned Value related discrepancies, no matter how minor, must be documented on a written CAR and address at a minimum the severity level of consequences if not corrected, and appropriateness of supplier corrective actions. The timely notification, prompt receipt by the due date, and disposition of a CAR is of the utmost importance in the resolution of system issues. This CAR process applies in what ever situation in which the discrepancy was discovered; regardless of role, process, procedure, or type of discovery. The surveillance team should report all systemic deficiencies to the EVM Center prior to the issuance of any CAR, regardless of severity. A deficiency exists when the design or operation of a supplier EVMS does not allow management or other personnel, in the normal course of performing their assigned functions, to have immediate access to reliable and accurate data and information for decision making purposes.

21

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008

EVMS CAR PROCESS

DCMA ASSESSMENT

DEFICIENCY IDENTIFIED

NO

YES

DCMA CAR GENERATED NO

DCMA APPROVAL

YES

CAR FORWARDED TO SUPPLIER

SUPPLIER DEVELOPS CORRECTIVE ACTION PLAN

CONTINUE ROUTINE SURVEILLANCE

NO

SUPPLIER SUBMITS CORRECTIVE ACTION PLAN TO DCMA

DCMA APPROVES CORRECTIVE ACTION PLAN YES DCMA TO EVALUATE PROGRESS TOWARDS COMPLETING CORRECTIVE ACTIONS NO DCMA APPROVES THE CAR TO BE CLOSED

YES

FIGURE_2. STANDARD CAR PROCESS 22

STANDARD SURVEILLANCE OPERATING MANUAL JAN 20089.1 ISSUING A CAR The issuance of a CAR involves quantitative and qualitative considerations, including: Number of discrepancies observed; Associated absolute dollar value impact; Importance of items to the accomplishment of contract requirements; and Potential impact on government funding.

Notwithstanding the above, the EVM Center makes the final decision as to whether or not a CAR will be issued and closed. Note: All CARs are generated, issued, and tracked through resolution and for trend analysis following the Corrective Action Request (CAR) Process as described herein. 9.2 CAR PURPOSE The purpose of a Corrective Action Request (CAR) is to formally notify the supplier that a documented course of action in the form of a Corrective Action Plan is needed to bring the EVMS in compliance with regulatory requirements. In some instances, DCMA may decide that the gravity of the deficiency and the suppliers response warrants an escalation of the issue. The suppliers response to past non-compliances should weigh heavily in this decision. Escalation brings higher visibility on both the government and supplier sides, bringing more focused attention to an otherwise disregarded and deficient issue. The CAR can be escalated by including recent history that warrants escalation, and increasing severity level of the CAR. Following severity level procedures, by default, increases level of visibility in the distribution. The severity level of the CAR dictates who receives the CAR in the suppliers organization. 9.3 CAR SEVERITY LEVEL

23

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008All CARs are coordinated within the surveillance team and approved by the EVM Center prior to issuance to the supplier. Coordination to the Division level is intended as a courtesy for information only, and does not imply approval authority. All EVMS CAR approval authority rests with the EVM Center, prior to being distributed to the supplier. All CARs will be documented and tracked. Verbal CARs are not acceptable. Level I CAR is issued when a contractual non-compliance requires no special management attention to correct. Level I CARs are directed to the supplier working level personnel. Level II CAR is a request for corrective action for contractual noncompliances that are systemic in nature and/or could adversely affect cost, schedule, or performance if not corrected. A Level I CAR may be escalated to a Level II CAR as the result of Level I CARs for the same types of non-conformances, across several programs/contracts or several Control Account Managers, indicating a systemic issue. Level II CARs are directed to the supplier management level responsible for the process with a copy to the responsible ACO. Level III CAR identifies issues where cost, schedule, technical performance, resources, or management process issues have unfavorably affected program performance and have not been corrected by the supplier. A Level III CAR need not be preceded by a Level I or Level II CAR. A CAR may also become a Level III after Level I and/or Level II attempts have failed and escalation is warranted OR in the case where the situation is deemed serious enough to warrant higher level attention. Failure to meet requirements cited in a CAR may include, but is not limited to, poor or incomplete corrective action plan, poor or missing root cause analysis, irreconcilable differences between DCMA and supplier. The

24

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008suppliers failure to appropriately correct a non-compliance in a Level III CAR shall result in an escalation from Level III to Level IV. A Level III CAR is addressed to the supplier's (site specific) top tier business manager. Level III CARs may be coupled with contractual remedies such as reductions of progress payments, cost disallowances, cure notices, show cause letters, or management systems disapprovals. Level III CARs may trigger formal reviews such as post award review for cause, compliance reviews, or other system validation reviews and may result in suspension or revocation of EVMS certification. When Level III CAR is closed, copies of the closed CAR should be sent to all those addressed and/or copied in the original CAR, as appropriate. Level IV CAR identifies issues where cost, schedule, technical performance, resources, or management process issues have unfavorably affected program performance across multiple programs or multiple sites; and have not been corrected by the supplier. A CAR also becomes a Level IV after Level III attempts have failed and escalation is warranted. The CAR should be addressed at the supplier's corporate level. A level IV CAR is issued to advise the supplier of contractual remedies such as suspension of progress payments or product acceptance activities, termination for default, and suspension or debarment, in accordance with applicable FAR/DFARS policies and procedures. Level IV CARs may trigger formal reviews such as post award review for cause, compliance reviews, or other system validation reviews and may result in suspension or revocation of EVMS certification. When a Level IV CAR is closed, copies of the closed CAR should be sent to all those addressed and/or copied in the original CAR, as appropriate.

25

STANDARD SURVEILLANCE OPERATING MANUAL JAN 20089.4 CAR SUBMITTAL All CARs should be submitted in a timely manner so as not to reduce impact of non-conformance observation. Within 5 working days of the non-conformance observation, the surveillance team will submit the CAR Part 1 to the EVM Center for review and approval. The 5 working days includes gathering of all required documentation as necessary (ex: review findings, discrepancy reports, routine surveillance communication). The surveillance team is responsible for ensuring that the CAR is properly coordinated with all concerned parties before it is submitted to the EVM Center for approval. The CMO Commander or designee must sign all CARs before formal submission to the EVM Center. Next, the EVM Center shall provide guidance in the way of approval / disapproval (Part 1) of the CAR within 5 working days and review the CAR for the following: 1. 2. 3. Correct severity level; Correct guideline; and Clear description of the non-compliance, guideline requirements, and supplier procedures If the EVM Center disagrees with any of the above criteria, it will be disapproved and sent back to the surveillance team for correction. For minor problems, the EVM Center may make corrections on-the-spot, at its discretion. Upon approval from the EVM Center, the surveillance team issues the CAR to the supplier. The following CAR Form shall be used (note: Digital CAR Form linked here has slight modifications).

26

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008PART 1. CAR Initiation 1.a. CAR Record # 1.b. CAGE 1.e. Initiator Name 1.i. Suppliers Name 1.l. Contract Number 1.f. Repeat? Select 1.j.Suppliers Location (city / state) 1.m. CPR DID on contract Select Dated: 1.k. Program Name 1.n. IMS DID on Contract Select Dated 1.r. EVM Clause # on Contract Select

1.c. Date Non-Compliance Observed 1.g. List Repeat CAR #s

1.d.Date CAR Initiated

1.h. Performance Commitments Impacted

1.o. Organizational Area 1.p. Guideline 1.q. Level of CAR Select Select Select 1.s. Supplier EVMS System Description Revision Number and Date 1.t. System Description Reference(Include Attachments if Necessary) 1.u. Description of Non-Compliance(Include Attachments if Necessary)

1.v. DCMA CMO Commander/Director or Designee (Printed Name and Signature) PART 2A EVM Center review 2.a. EVM Center Product Lead 2.c. EVM Center Product Lead Comments

1.w. Date Sent to EVM Center RevisionSelect

2.b. Date Reviewed

2.d. EVM Center Action - Returned to CMO for 1) Submittal to Supplier (if concur); 2) Rework (if non-concur) CONCUR NON CONCUR PART 2B. CMO Submittal to Supplier 2.f. Date Submitted to Supplier 2.e. Date: 2.g. Supplier Response Due Date Revision Select

PART 3. Supplier Response 3.a. Supplier Response Date 3.b. Supplier Response and Corrective Action Plan (CAP)

PART 4. Disposition (CMO Working Through EVM Center) 4.a. APPROVED CONDITIONALLY APPROVED 4.b. DCMA Disposition Comments 4.c. CAP Implementation Date 4.f. DCMA Verification Comments 4.g. Escalation / Disposition Options Approved/Closed Withdrawn/Cancelled Write Further CAR Observations Escalate/Increase CAR level Request Management System Review from EVM Center 4.d. DCMA verified? Select

Revision Select DISAPPROVED

4.e. Verification Date

4.h. Disposition Date

27

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008EVM Center Final Concurrence (MUST BE CHECKED)

After a Level III or Level IV CAR has been approved by the EVM Center, the surveillance team working through the CMO will immediately provide copies to: 1. 2. 3. Buying command Program Management Office; Applicable DCMA Divisions; and Appropriate Contract Integrity Center Counsel (DCMA Legal department). The surveillance team is responsible for: 1. Ensuring that any questions the supplier has regarding the CAR are answered promptly. Difficult questions requiring further guidance should be coordinated with the DCMA EVM Center; 2. 3. 4. 5. 6. Follow up with the supplier to ensure suspense dates are met; Providing preliminary answers to corrective action plan questions from supplier; Keeping all interested parties informed as to the status of the CARs; Knowing the status of the CAR and corrective action plans by keeping a tracking log; and Performing and maintaining a CAR trend analysis.

At a minimum, the surveillance team must track the following: 1. 2. 3. 4. 5. 6. 7. 8. Date when CAR given to supplier; Requested Due Date of suppliers response Person who initiated the CAR; EVM Center point of contact; Suppliers POC for the CAR; Actual date of suppliers response; Root cause of non-compliance; Date corrective action plan received;

28

STANDARD SURVEILLANCE OPERATING MANUAL JAN 20089. Suppliers corrective action;

10. Date verification performed by DCMA; 11. Final CAR disposition status; 12. Final CAR disposition status date; 13. Repeat non-compliances; and 14. Date corrective action plan approved/disapproved. At a minimum, the surveillance team must track the following for trending purposes: 1. 2. 3. 4. Number of EVMS CARs by supplier and location; Number of repeat EVMS CAR non-compliances by supplier; Number of EVMS CARs by guideline by supplier; and Number of days each EVMS CAR remains open, by level and supplier.

9.5 DISPOSITION The suppliers response to the CAR may include a corrective action plan (CAP). Upon receipt of the suppliers response, the surveillance team, working through the CMO, performs a preliminary review of the corrective action taken or planned. The review will consist of the following: 1. 2. 3. 4. Good root cause analysis; Good corrective action to prevent recurrence; Any repeat non-compliances; and Guideline compliance.

Based on the review of the suppliers response, the CMO, working through the EVM Center, has approval/disapproval/conditional approval of the CAP. This shall occur within 5 working days. If the preliminary review finds deficiencies in the CAP, the surveillance team, working through the CMO, will reject the corrective action plan and return it to the supplier for resolution. If the surveillance team finds no deficiencies within the CAP, it will then be given to the EVM Center for review. 29

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008

The EVM Center will also review the corrective action plan for the following: 1. 2. 3. 4. Good root cause analysis; Good corrective action to prevent recurrence; Any repeat non-compliances; and Guideline compliance.

Approval If no CAP is required, the surveillance team, working through the EVM Center, may approve the CAR. Conditional Approval When a CAP is required, the CAR may be conditionally approved, contingent upon verification. Conditional approval indicates that the CAP appears acceptable but further verification is necessary to ensure systemic incorporation over the course of time. Disapproval The CMO documents why the CAP or CAR is being disapproved and submits to the supplier as rejected for a resolution. If unresolved discrepancies arise between corrective action plan expected results and DCMA (CMO and EVM Center) expected results, then escalation steps ensue. The surveillance team continues tracking the status of the corrective action plan until final disposition is reached. The EVM Center makes the determination of the necessity of verification follow up actions. The EVM Center also approves all verification follow up actions. The decision for verification is based on the following: 1. 2. 3. Gravity of the non-compliance; On-site visual inspection required to determine if the supplier is actually doing what the corrective action plan says; Corrective action plan effectiveness towards satisfying the guideline(s); and

30

STANDARD SURVEILLANCE OPERATING MANUAL JAN 20084. Previous disagreements, previously disapproved actions in the CAR process, or lingering doubt about guideline compliance. If verification follow up is necessary, the surveillance team working through the DCMA EVM Center writes closure criteria. The surveillance team is responsible for ensuring that the closure criteria are followed, and that a mutual understanding has been reached. Verification may consist of reviewing the completeness of any of the products and data that are required for each of the guidelines. If the surveillance team working through the EVM Center determines that verification is not necessary, then the CAR is returned to the surveillance team and closed out. Any questions regarding the instructions should be directed to the EVM Center. Verification status is tracked by the surveillance team. The closure criteria should contain clear activities required to be successfully accomplished before the CAR can be closed out. The surveillance team documents the status of these activities and is responsible for ensuring that the statuses of activities are documented. The surveillance team keeps the EVM Center informed of verification status. If problems arise, the EVM Center must be notified immediately. Final Disposition The EVM Center must approve the CAR by checking block 4.g. EVM Center Final Concurrence before it is officially closed out by the CMO. Before suggesting a CAR for close out, the surveillance team must answer the following close out evaluation questions. 1. 2. 3. 4. 5. Is the guideline being met? How is this different from when the guideline was not being met? Will the guideline be met in the future? Does this CAR affect the supplier being compliant with other guidelines? and Are other projects affected by the CAR? If so, will they be compliant with the guidelines? 31

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008

If the answer is unsatisfactory to any of these questions, then the EVM Center will decide if an escalation should ensue or disapproval should follow. If no escalation is considered necessary then the surveillance team may be asked to support the following questions and appropriate course of action: 1. 2. 3. Is more verification necessary by the CMO? Is a different type of verification necessary? and Is a new or modified corrective action plan required to be submitted by the supplier? If the answer is satisfactory to the close-out evaluation questions, then the EVM Center writes a short narrative describing that approval conditions have been met and sends it to the surveillance team working through the CMO for closeout. 10.07

CAR CENTRAL REPOSITORY (AT PRESENT UNDER CONSTRUCTION) A central electronic repository is being developed to contain CAR-related data for all programs, contracts, and suppliers with EVM requirements. Setting up a central repository has the advantages that a single repository can display data from other components. To create a central repository, each surveillance team must register the CAR with the central repository regardless of severity level. By registering standard CAR information on supplier system deficiencies implementation, issues can be identified and remediated through the EVM Center.

11.08

DOCUMENT AND REPORT RESULTS The surveillance team ensures that documentation of all surveillance related activities, including surveillance meeting minutes, surveillance reports, and supplier activities related to EVMS surveillance are retained and stored for centralized availability. The CMO will generate a written report that details the findings and recommendations. The CMO objective in the surveillance of the

32

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008supplier EVMS is to express an opinion on DCMAs collective assessment of the effectiveness of EVMS implementation. 11.1 DOCUMENTATION RESPONSIBILITIES The surveillance report is a documented record capturing all aspects of the surveillance review. The report shall be submitted to the Center no later than 10 business days after the final surveillance assessment in accordance with the reporting period stated in the approved Annual EVMS Surveillance Schedule. After review, the EVM Center will forward opinions and recommended changes to the surveillance team no later than 5 business days after receipt.

PERFORMANCE TASK 1. The surveillance team generates and submits surveillance report and submits it to the EVM Center for review. 2. The surveillance team working through the EVM Center initiates the CAR process, if necessary. 3. The EVM Center submits comments and direction to the surveillance team. 4. Working through the EVM Center, the surveillance team updates and submits the final report to the ACO, PI, PMO, and supplier.

DURATION 10 business days Immediate 5 business days 5 business days

Minimum expectations for documentation to be addressed in the System Surveillance Report (SSR) include: Surveillance Selection Risk Matrix(s); Guidelines or Process(es) reviewed; PM and CAM(s) interviewed and Control Accounts examined; Completed Guideline templates for each reviewed Guideline

EV Templates 1-32 Completed EVMS Description Cross Reference Checklist; and

System deficiencies identified: 33

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008 Corrective Action Request(s); Supplier Corrective Action Plan in place; Actions taken to correct the deficiency; and CMO analysis for trends and systemic issues. 11.2 ADMINISTRATIVE CONTRACTING OFFICER (ACO) RESPONSIBILITIES Per DFARS 252.234-7002 the Cognizant Federal Agency is the authority for recognizing the supplier EVMS as either compliant or non-compliant with the 32 ANSI/EIA-748 EVMS guidelines as stipulated by the contract. In DCMA, the ACO issues an AA or LOA indicating system acceptability. A LOA is prepared when a supplier does not wish to enter into a longer term AA with DCMA. A LOA is issued for non-DoD suppliers having gone through the DCMA EVMS validation process but elect not to have a follow-on DCMA surveillance presence. Conversely, an AA demonstrates that a DoD supplier has successfully completed the validation process, has entered into a joint surveillance plan, and is committed to using the EVMS as part of its management process. If changes occur to a supplier EVMS with an AA or LOA in place, the CMO working through the EVM Center will review and consider all proposed changes to ensure for compliance with the 32 ANSI/EIA-748 EVMS guidelines. If surveillance, as described herein, deems the supplier EVMS to be non-compliant, the EVM Center will make recommendations to the ACO. Per the DCMA EVM Concept of Operations, August 2006, the ACO is expected to seek direction from the EVM Center regarding all matters related to EVM and EVMS. As part of the yearly system surveillance/compliance verification process, the cognizant ACO is required to formulate and submit to the EVM Center a status of the suppliers EVMS compliance, including all supporting data to that effect, such as the annual recommendation from the EVM Specialist. This information will be used by the EVM Center to update the EVMS Supplier Validation list. It is the EVM Centers responsibility to ensure that the information is accurate and

34

STANDARD SURVEILLANCE OPERATING MANUAL JAN 2008updated annually or when the status of a supplier system changes, whichever is earlier. 12.09

ESTABLISH AND MAINTAIN SURVEILLANCE FILES Surveillance files are established and indefinitely maintained by the CMO to hold all pertinent data and information, including surveillance plans and surveillance findings, recommendations and actions.

13.010

UPDATE STANDARD SURVEILLANCE PLAN AS NECESSARY As supplier work scope or contracts change, the SSP should be updated accordingly. If, for example, a supplier that once did not have a DFARS EVM requirement but due to circumstances now has a DFARS EVM requirement, a SSP following the steps described herein is expected to be developed. Adjustments to the SSP must be approved by the EVM Center prior to the implementation of any changes.

35

STANDARD SURVEILLANCE PLAN JAN 2008

DCMACITY, STATE

STANDARD SURVEILLANCE PLAN[DATE]36

STANDARD SURVEILLANCE PLAN JAN 2008INTRODUCTION This plan is issued to provide a uniform way to ensure that an Earned Value Management System (EVMS) meets the 32 ANSI/EIA-748 EVMS guidelines. The acceptance of this plan signifies that the DCMA and [SUPPLIER NAME] have entered into an agreement to ensure that [SUPPLIER NAME] [SUPPLIER SITE] has implemented a compliant EVMS, as documented in the [SUPPLIER NAME] [SYSTEM DESCRIPTION TITLE] [DATE].

DCMA has an active surveillance program designed to promote a common understanding of the expectations for compliance with the requirements of the 32 ANSI/EIA-748 EVMS guidelines. Compliance with the guidelines helps to ensure consistent and comparable reporting between entities, which is vital to confidence in the EVMS. The supplier team members must be independent of the management chain of the programs that it is responsible for surveying. Although a number of stakeholders are involved in the surveillance of the EVMS, the DCMA in accordance with DFARS1 requirements and its role as DoDs Executive Agent for EVMS has primary responsibility for overseeing suppliers implementation. Suppliers are encouraged to be active participants in the surveillance review process. The review format is not intended to replace the suppliers internal EVMS surveillance process or in any way remove the suppliers responsibility to implement a compliant EVMS. The goal of the surveillance review process is to reduce the duplicative efforts and cost of surveillance by combining resources to achieve common goals. Responsibilities of the surveillance team include: 1

Developing an annual surveillance plan and approach; Appointing a DCMA team leader for program surveillance reviews;

DFARS Subpart 242.302(a)(S-71) assigns the Contract Management Office (CMO) function for EVMS to the DCMA, including the responsibility for reviewing EVMS plans and verifying initial and continuing compliance with DoD EVMS criteria.

37

STANDARD SURVEILLANCE PLAN JAN 2008 Assigning resources to the surveillance reviews; Communicating the results of the surveillance; Tracking surveillance findings to closure; Developing and maintaining surveillance databases and metrics to assess the systemic health of the EVMS processes, as assessed across multiple reviews; Recommending EVMS process implementation and training changes to correct systemic findings.

I.

PURPOSE AND OBJECTIVES The surveillance review process is established for the following reasons:

1.

Assess the suppliers commitment and ability to implement the EVMS as an integral part of its management process and to ensure that cost and schedule reports provide the customer with: Timely and reliable cost, schedule, and technical performance measurement data and information that depicts actual conditions; Data and information derived from the same database as that used by the supplier for the management of the program; Data and information that is auditable; Timely indications of actual or potential problems; Comprehensive variance analysis and corrective action reporting regarding cost, schedule, technical, and other problem areas, as well as proposed date(s) for cost and schedule recovery; and program. Insights on actions taken to mitigate risks to the

2.

Ensure that the supplier EVMS continues to be compliant with the 32 ANSI/EIA748 EVMS guidelines by:

38

STANDARD SURVEILLANCE PLAN JAN 2008

Training designated program personnel in the use of the EVMS; Accomplishing early, comprehensive planning to provide a quality baseline ready for examination in the Integrated Baseline Review (IBR) process;

Integrating cost, schedule, and technical planning into a single, wellcontrolled performance measurement baseline.

Establishing clear lines of authority and responsibility for accomplishment of work elements;

Using information early, and continuously, to formulate corrective actions and work around plans to mitigate significant variances from the baseline plan;

Providing valid and timely management information; and Ensuring for the integration of management systems.

3.

Encourage continuous improvement and innovation of the EVMS to include people, processes, tools, and techniques.

4.

Maintain a disciplined process using EVM, including effective teamwork between the government and supplier.

5.

Effectively communicate surveillance findings and results, including areas where the supplier demonstrates ineffective use of the EVMS.

6. 7.

Document those findings on corrective action requests. Follow-up on the contractor's corrective action to assure the current and any foreseeable problems are eliminated.

8.

Maintain metrics to determine the effectiveness of the EVMS and to distinguish between systemic and non-systemic problems.

9.

Reduce the cost of surveillance by combining resources to achieve common goals.

39

STANDARD SURVEILLANCE PLAN JAN 2008

II.

6 STEP SURVEILLANCE PROCESS STEP 1. SURVEILLANCE SCOPE The EVMS surveillance process is risked-based and assessed annually, with a requirement to review all 9 management processes and 32 guidelines over the course of a year. This allows flexibility in the timing of scheduled reviews and adjusting for key program events so that surveillance does not intrude on program requirements yet appropriately matches process reviews with program content. The selection of key management processes and guidelines reviewed should be relevant to the program phase and provide an opportunity for coaching or mentoring during the process review.

The scope of each surveillance review includes a comprehensive assessment of the supplier EVMS compliance with the 32 ANSI/EIA-748 EVMS guidelines and implementation in accordance with descriptive documents. The surveillance team is responsible for documenting the findings relevant to the key management processes and guidelines. Through data traces and manager interviews, the surveillance team will assess use of EVMS data and documentation in the operation of the programs. The surveillance team will make final recommendations regarding compliance with the 9 processes, 32 guidelines, and all aspects of the EVM operation. The team will determine: 1. Whether processes, procedures, and methods are compliant with the EVMS guidelines; 2. Whether descriptive documents containing contractor's policies and procedures are understood and followed in actual operation; 3. 4. How the data is generated by the system; How the data is used in the management of the program; and

40

STANDARD SURVEILLANCE PLAN JAN 20085. Managements knowledge of the EVMS roles and responsibilities of its operating personnel.

STEP 2. SURVEILLANCE METHODOLOGY Notification: The surveillance team will provide adequate advanced notification of specific control accounts and processes that will be reviewed. It also will provide the program adequate notice to ensure that access to documentation, facilities, and resources will not interfere with critical time sensitive work.

Risk-Based Approach: EVMS risks are identified by using the Surveillance Selection Risk Matrix for each EVMS program/contract (Attachment 1). The review schedule will include all processes, with more intense reviews on those programs/contracts with high or medium risk since they are most likely to cause unfavorable cost, schedule, and technical performance impacts.

Program Documentation: To prepare for the on-site review, the surveillance team will gather and review both system and program documentation as well as perform data trace analysis. The supplier will provide the documentation no later than three weeks prior to the on-site review date. Depending on the process being reviewed, the surveillance team will request:

Program specific instructions on EVMS implementation; Correspondence relating to EVMS; Organization charts; Statement of Work; Contract Work Breakdown Structure; Dollarized Responsibility Assignment Matrix identifying Control Account Managers by WBS and OBS;

41

STANDARD SURVEILLANCE PLAN JAN 2008

Work authorization documentation; Contract Budget Baseline, Management Reserve, and Undistributed Budget logs;

Control Account Plans; Material purchasing reports; Subcontractor reports, as applicable; Contract Performance Reports; Program schedules, Integrated Master Schedule; EVM related contract deliverables; Staffing plans; Rate applications and changes since the last review; Modifications to the contract since last review; and Estimate at Completion supporting documentation.

NOTE: The previous list is not exhaustive and can be updated to include more specific items, or can be tailored to remove items not required at any one specific review. If the surveillance team determines that more information or a greater level of detail is required, the supplier will provide that data and information no later than two weeks prior to the on-site review.

On-Site Review: The on-site review includes an in-brief, discussions with key supplier program personnel, and an out-briefing.

The on-site review process begins with an in-brief during which the surveillance team describes the objectives of the surveillance process and the approach used to assess the implementation and use of EVMS. The supplier is expected to

42

STANDARD SURVEILLANCE PLAN JAN 2008provide relevant program information including but not limited to each programs technical objectives, period of performance, critical subcontractor performance, major achievements, current issues, and upcoming key event milestones. The inbrief should be attended by the entire surveillance team.

Interview discussions are held with key supplier personnel including the Program Manager(s), Business Manager(s), Control Account Managers (CAMs), Project Controls personnel, and other key program team members. The surveillance team will identify those to be interviewed based on the off-site documentation review. Interviewees are expected to demonstrate knowledge and use of the EVMS.

Time is built into the on-site review in order for the surveillance team to discuss interview results, findings of deficiency, areas for improvement, and best practices. The results of these discussions form the basis for Corrective Action Requests (CARs), out-briefing, and final surveillance reports.

The on-site review process concludes with an out-briefing during which the surveillance team presents all CARs approved for issue and best practices identified in the review. For each CAR generated, the surveillance team will identify: severity level, specific guidelines and processes affected, and risk to the program. In response to each CAR, the supplier will identify the required corrective action, the responsible individuals for the correction, and a potential planned completion date. If the supplier is unable to provide corrective action information at the out brief, it will be provided to the surveillance team following the DCMA standard CAR process (block 2.g of the CAR form).

STEP 3. SURVEILLANCE TEAM

43

STANDARD SURVEILLANCE PLAN JAN 2008The surveillance team consists primarily of individuals from the DCMA, supplier, and, when appropriate, the DCAA. Additional participants may come from the Government Program Management Office (PMO), other departments within the DCMA, and the supplier. All participants are expected to be experienced in the surveillance process, knowledgeable in the application of the 32 ANSI/EIA-748 EVMS guidelines, and familiar with the suppliers EVMS documentation and processes.

STEP 4. COMMUNICATIONS AND COOPERATION The PMO(s) and supplier will recognize that the surveillance team is an integral part of the management process. The team will openly communicate its findings and concerns with the PMO and supplier. The surveillance review results should be discussed on an on-going basis, including the identification of deficiencies and the status of corrective actions at monthly program management meetings. The supplier will keep the surveillance team advised of planned or actual changes to the EVMS prior to the implementation of the change, including, changes in software tools, key processes, or internal management procedures. The surveillance team will note the severity of the changes and may recommend a system review. Any changes to an already approved EVMS must be approved by the EVM Center.

To facilitate the surveillance review the supplier commits to the following:

Identification of an on-site review coordinator Adherence to agreed upon surveillance schedule Pre-coordinated security and facility entrance requirements Access to meeting room equipped with audio/visual capability, Appropriate program staff availability and attendance Timely documentation delivery

44

STANDARD SURVEILLANCE PLAN JAN 2008

Timely response to any requests for additional information Uninterrupted interviews with program personnel Access to printing and copying equipment, as needed

STEP 5. SURVEILLANCE SCHEDULE The surveillance team will establish an annual EVMS surveillance schedule for performing surveillance reviews on selected programs/contracts. Periodic surveillance team meetings will be held to discuss EVMS metrics, results from system and program surveillance activities, results from data trace analysis and Integrated Baseline Reviews, and concerns raised by government users of the EVMS.

The surveillance schedule will include a sufficient sample of programs/contracts so that a credible assessment of the suppliers implementation and use of the EVMS can be made. The system surveillance schedule (attachment 2) is developed annually and identifies processes, guidelines, program(s)/contract(s), and timeframes. The surveillance team will update the surveillance schedule based on risks and new developments, as appropriate. For the last month of the year, the surveillance team will conduct a reconciliation of reviews and CARs, and provide a summarization with a recommendation relative to the status of the EVMS to the ACO.

STEP 6. SURVEILLANCE FINDINGS AND RESOLUTION The surveillance team will conclude the review with an out-briefing during which the surveillance team will present the surveillance results to relevant stakeholders. The surveillance team will document the results of the review in a written report that is signed and dated by the DCMA surveillance team leader and supplier representative. The report will be issued in accordance with the annual

45

STANDARD SURVEILLANCE PLAN JAN 2008EVMS surveillance schedule and should include an overall assessment of the suppliers implementation of the EVMS, scope of the review, personnel interviewed, and findings of deficiency or non-compliance that resulted in CARs. The surveillance team will use the CAR process for the initiation and follow up of correcting system deficiencies.

All CARs will be tracked and managed until resolution. If an identified compliance issue or deficiency cannot be resolved by the surveillance team and supplier the unresolved issue shall be elevated to the EVM Center for resolution. Should the supplier not make adequate or timely progress in correcting identified noncompliances or deficiencies, financial and system remediation actions may be initiated to protect the Governments interest, including suspending or withdrawing the EVMS validation.

This Standard Surveillance Plan remains in place indefinitely, subject to modification by mutual agreement or termination by either party. The Standard Surveillance Schedule will be updated annually.__________________________________ [Supplier name] [Supplier site] Date:_______________________

__________________________________ [CMO surveillance specialist name] CMO location

Date:_______________________

__________________________________ [EVM Center Product Lead name] EVM Center, DCMAC-V

Date:_______________________

46

ATTACHMENT 1 SURVEILLANCE SELECTION RISK MATRIX JAN 2008SURVEILLANCE SECTION RISK MATRIX SUPPLIER: RISK WEIGHT PROGRAM .05 PHASE PM EVM EXPERIENCE TOTAL CONTRACT VALUE VALUE OF PRIME WORK REMAINING VALUE OF SUBC WORK REMAINING .05 DATE: PROGRAM: HIGH 3.00 DEVELOPMENTORGANIZING, SCHEDULING, WORK/BUDGET AUTHORIZATION

POC: CONTRACT: LOW 1.00 LATE LRIPMATERIAL MANAGEMENT, ACCOUNTING

MEDIUM 2.00 EARLY LRIPACCOUNTING, MATERIAL MANAGEMENT, CHANGE INCORPORATION

SCORE .15

< 2 YRSORGANIZING, SCHEDULING, MANAGERIAL ANALYSIS

2 5YRSSCHEDULING, MANAGERIAL ANALYSIS

> 5YRSMANAGERIAL ANALYSIS

.05

.05

> $99MWORK/BUDGET AUTHORIZATION, ACCOUNTING, MANAGERIAL ANALYSIS

$50M - $99MSCHEDULING, WORK/BUDGET AUTHORIZATION

$20M - $49MSCHEDULING

.15

.10

> 50%MANAGERIAL ANALYSIS, CHANGE INCORPORATION

10 - 50%MANAGERIAL ANALYSIS, CHANGE INCORPORATION

< 10%ACCOUNTING, MATERIAL MANAGEMENT

.20

.10

> 50%WORK/BUDGET AUTHORIZATION, SCHEDULING, SUBCONTRACT MANAGEMENT, MANAGERIAL ANALYSIS

10 50%WORK/BUDGET AUTHORIZATION, SCHEDULING, SUBCONTRACT MANAGEMENT, MANAGERIAL ANALYSIS

< 10%ACCOUNTING, SUBCONTRACT MANAGEMENT

.30

VALUE OF MATERIAL REMAINING VALUE OF MGMT RES REMAINING OTB (RESETS)

.10

>30%WORK/BUDGET AUTHORIZATION, SCHEDULING, ACCOUNTING, MATERIAL MANAGEMENT

15 30%ACCOUNTING, MATERIAL MANAGEMENT

< 15%MATERIAL MANAGEMENT

.30

.05

< 5% BCWRWORK/BUDGET AUTHORIZATION, CHANGE INCORPORATION

5 10% BCWRWORK/BUDGET AUTHORIZATION, CHANGE INCORPORATION

> 10% BCWRCHANGE INCORPORATION

.10

.05

2 or moreWORK/BUDGET AUTHORIZATION, CHANGE INCORPORATION, SCHEDULING

1WORK/BUDGET AUTHORIZATION, CHANGE INCORPORATION

NILORGANIZING

.10

SV%, CV%, OR VAC% CRITICAL PATH FLOAT

.05

> 10%ACCOUNTING, INDIRECT MGMT, MANAGERIAL ANALYSIS

5 - 9%INDIRECT MANAGEMENT, MANAGERIAL ANALYSIS

< 5%MANAGERIAL ANALYSIS

.15

.10

NEGATIVE NO (0) MARGINSCHEDULING, MANAGERIAL ANALYSIS

POSITIVE 40 POSITIVE WORK DAYSSCHEDULING, WORK/BUDGET AUTHORIZATION

.30

BASELINE VOLATILITY INDIRECT RATES

.05

> 15%CHANGE INCORPORATION, ACCOUNTING

5 - 15%CHANGE INCORPORATION, ACCOUNTING

< 5%MANAGERIAL ANALYSIS

.10

.05

NO FPRAINDIRECT MANAGEMENT, ACCOUNTING

PROPOSED FPRAINDIRECT MANAGEMENT, ACCOUNTING

APPROVED FPRAINDIRECT MANAGEMENT

.10

ONGOING SYSTEMS ISSUES TIME SINCE LAST REVIEW TOTAL

.15

MULTIPLE UNRESOLVEDAFFECTED PROCESSES

SINGLE UNRESOLVEDAFFECTED PROCESSES

NILNA

.45

.05

>12 MO. OR NEVER REVIEWEDALL PROCESS GROUPS

6 -12 MO.PROCESSES NOT YET REVIEWED

< 6 MO.FOLLOW ALL OF THE ABOVE

.15

1.00

2.60

47

ATTACHMENT 2 ANNUAL EVMS SURVEILLANCE SCHEDULE JAN 2008ANNUAL EVMS SURVEILLANCE SCHEDULE PART 1: 1.A. CALENDAR YEAR 1.B.CMO POINT OF CONTACT

20081.C. CMO NAME

CAPTAIN ROGER PETERSON1.D. CMO LOCATION (CITY/STATE)

SPACE AND MISSILES1.E. SUPPLIER NAME

TAMPA, FL1.F. SUPPLER LOCATION (CITY/STATE)

MISSILE-MART, INC.1.G. DATE SURVEILLANCE SCHEDULE PREPARED

TAMPA, FL1.H. DATE SCHEDULE APPROVED BY EVM CENTER

DECEMBER 15, 2007

DECEMBER 31, 2007

PART 2: FOR EACH OF THE NINE PROCESSES, IDENTIFY WHICH GUIDELINES WILL BE REVIEWED (REF. EVMIG FIGURE 2-1 GUIDELINESPROCESS MATRIX). BASED ON THE SURVEILLANCE SELECTION RISK MATRIX RESULTS, IDENTIFY WHICH PROGRAM(S) /CONTRACT(S) WILL BE REVIEWED AGAINST EACH OF THESE GUIDELINES, WHEN THE SURVEILLANCE ACTIVITIES WILL OCCUR, AND LASTLY, WHEN REPORTS WILL BE COMPLETED AND RELEASED. 2.A. PROCESS(ES) 2.B. GUIDELINES 2.C. PROGRAM(S) / CONTRACT(S) 2.D. PERIOD OF SURVEILLANCE 2.E. SCHEDULED COMPLETION DATE OF REPORT (SSR)

ORGANIZING

1, 2, 3, 5

ABC/N00104-07-CXXXX; XYZ/DAAB07-07-CZZZZ ABC/N00104-07-CXXXX; XYZ/DAAB07-07-CZZZZ ABC/N00104-07-CXXXX; XYZ/DAAB07-07-CZZZZ ABC/N00104-07-CXXXX; XYZ/DAAB07-07-CZZZZ DEF/F33657-05-CYYYY XYZ/DAAB07-07-CZZZZ DEF/F33657-05-CYYYY TUV/00NAS8-06XXXX DEF/F33657-05-CYYYY TUV/00NAS8-06XXXX DEF/F33657-05-CYYYY

JANUARY 2008

JANUARY 2008

WORK/BUDGET AUTHORIZATION

8, 9, 10, 11

FEBRUARY 2008

COMBINE WITH MARCH 2008 MARCH 2008

WORK/BUDGET AUTHORIZATION

12, 14, 15

MARCH 2008

SCHEDULING

6 AND 7

APRIL 2008

APRIL 2008

ACCOUNTING

16, 17, 18, 20, 22, 30 21 (2, 16)

MAY 2008

MAY 2008

MATERIAL MANAGEMENT SUBCONTRACT MANAGEMENT MATERIAL MANAGEMENT SUBCONTRACT MANAGEMENT MANAGERIAL ANALYSIS

JUNE 2008

COMBINE WITH JULY 2008 JULY 2008

(9, 10, 12, 22, 23, 27) (9, 10, 12, 22, 23, 27) 22, 23, 25, 26

JULY 2008

AUGUST 2008

COMBINE WITH

48

ATTACHMENT 2 ANNUAL EVMS SURVEILLANCE SCHEDULE JAN 2008TUV/00NAS8-06XXXX MANAGERIAL ANALYSIS 27 DEF/F33657-05-CYYYY TUV/00NAS8-06XXXX DEF/F33657-05-CYYYY TUV/00NAS8-06XXXX ABC/N00104-07-CXXXX; TUV/00NAS8-06XXXX SEPTEMBER 2008 SEPTEMBER 2008 SEPTEMBER 2008

CHANGE INCORPORATION

28, 29, 30, 31, 32

OCTOBER 2008

OCTOBER 2008

INDIRECT MANAGEMENT

4, 8, 13, 19, 24, 27

NOVEMBER 2008

NOVEMBER 2008

PART 3: END OF YEAR RECONCILIATION OF REVIEWS, PREPARE RECOMMENDATION TO ACO. 3.A. SCHEDULED COMPLETION DATE OF YEAR END RECOMMENDATION TO ACO DECEMBER 31, 2008

49