Closeout Presentation LBNF CD-3a EVMS Implementation Review · Closeout Presentation LBNF CD-3a EVMS Implementation Review January 11-12, 2017 Page 6 of 16 2.0 Introduction The purpose

Closeout Presentation

LBNF CD-3a

EVMS Implementation Review

January 11-12, 2017


LBNF CD-3a EVMS Implementation Review January 11-12, 2017

Page 2 of 16

This page intentionally left blank



Page 3 of 16

Table of Contents

Table of Contents .............................................................................................................................................. 3

1.0 Executive Summary ............................................................................................................................... 5

2.0 Introduction ............................................................................................................................................ 6

3.0 EVMS Implementation ........................................................................................................................... 7

3.1 EVMS Processes Including Cost and Schedule ...................................................................................... 7

4.0 Appendices ........................................................................................................................................... 12



Page 4 of 16

This page intentionally left blank



Page 5 of 16

1.0 Executive Summary

An Earned Value Management (EVM) Implementation Review of the Long-Baseline Neutrino Facility

(LBNF) Project was held on January 11-12, 2016 at the Fermi National Accelerator Laboratory (Fermilab).

At the request of Marc Kaducak, Head of Office of Project Support Services, the purpose of the review is to

assess the progress and plans for execution of the EVM processes on the CD-3a scope for the LBNF Project,

in accordance with standard DOE and Laboratory requirements. LBNF is a pre CD-2 project and not subject

to the full DOE Order 413.3B project EVMS processes. Nonetheless, it is managed and is evaluated according

to standard project management practices and has been maintaining a practice preliminary baseline. The focus

of this review is to ensure the implementation of the Fermilab EVMS on the LBNF Far Site Conventional

Facilities (FSCF) practice preliminary baseline and determine if the project team is ready to effectively

monitor and manage cost, schedule, and technical performance of the CD-3a portion of the FSCF upon award

of the first construction package. Only this area of scope was assessed for this review.

Overall, the committee found that the project team is committed to implementing EVM practices and

processes. The project team presented their own lessons learned based on evaluation of their preliminary

practice baseline, which is encouraged to continue. The Committee determined that there are two main areas

where additional improvements will be needed: reviewing issues identified with Work Authorization

Documents (WADs) prior to implementing a formal baseline and application of already identified lessons

learned once the formal baseline is established. By addressing these recommendations, the LBNF FSCF

project scope should be well placed to successfully set a baseline and begin official reporting.

The Committee identified additional areas of improvement – in the form of comments – that relate primarily

to performing within an EVMS more effectively. These, while important, are not necessarily preventing the

project from proceeding to establishing a successful baseline for reporting. Practice related to the EVMS

should continue to allow for a successful baseline establishment and reporting of the entire LBNF project.



Page 6 of 16

2.0 Introduction

The purpose of the review is to assess the progress and plans for execution of the EVM processes on the CD-

3a scope for the LBNF Project, in accordance with standard DOE and Laboratory requirements. This review

is to satisfy a recommendation from the DOE’s CD-3a Independent Project Review in December 2015 that

the project successfully complete an EVM implementation review prior to award of the first construction sub-

package.

This closeout report is broken down into two basic sections. The first section of this presentation has the

Committee’s answers to the review’s charge question. The second section provides the assessments of the

project’s EVM implementation and related data. Each area within this first section is organized by Findings,

Comments and Recommendations. Findings are statements of fact that summarize noteworthy information

presented during the review. The Comments are judgment statements about the facts presented during the

review and are based on reviewer's experience and expertise. The comments are to be evaluated by the project

team and actions taken as deemed appropriate. Recommendations are statements of actions that should be

addressed by the project team.

The LBNF Project is encouraged to develop a response to the assessment recommendations. The

recommendations should be tracked to closure and a documented status of the project’s resolution should be

available for future reviews.



Page 7 of 16

3.0 EVMS Implementation

3.1 EVMS Processes Including Cost and Schedule

Committee Lead: Jennifer Fortner

Committee Members: Sally Anderson, Sherese Humphrey, Fran Leavell

Charge Questions:

● Is the project team knowledgeable of its EVM roles and responsibilities and have they been trained on

FRA’s EVM processes and procedures? Yes, the LBNF FSCF project team members appeared

knowledgeable and have been trained regarding EVM.

● Has the project established and maintained a preliminary baseline for the Far Site Conventional

Facilities satisfying the requirements of FRA EVMS System Description (SD) and procedures as they

pertain to ANSI Standard 748-B? Yes, while there are areas of improvement identified below, the

preliminary baseline appears to have functioned in a way to allow the project team to learn and practice

EVM prior to full implementation.

● Is the project following FRA’s EVM requirements and standards? Yes, however, there are areas of

improvement and compliance identified below.

● Are Corrective Actions from the FRA March 2016 annual surveillance review being addressed and

implemented by the LBNF project team? Yes, while not all corrective actions are fully implemented

they have been identified as issues or lessons learned and will be implemented going forward.

● Are monthly reports being used to inform project stakeholders and shape advanced planning? Yes, the

project is using monthly reports for information and resulting corrective actions to advance planning.

● Are corrective actions being identified and improvements implemented in forward planning? Yes,

while not all corrective actions are fully implemented they have been identified as issues or lessons

learned and will be implemented going forward.



Page 8 of 16

● Is change control being properly administered in accordance with FRA’s requirements? Yes, however,

there are areas of improvement identified below.

● Is the project ready to implement a baseline on the CD-3a scope of work prior to award of the first

construction package? Yes, with implementation of the items identified below. It should be noted that

upon award of the construction package, changes to the baseline will occur to allow for the

incorporation of the contract awards. This is understood by the project team.

Findings

● An Organizational Breakdown Structure (OBS) was presented for the entire Long Baseline Neutrino

Facility/Deep Underground Neutrino Experiment (LBNF/DUNE) project including specifics on the

Far Site Conventional Facilities (FSCF) scope. This included internal and external organizations

responsible for accomplishing work.

● A Responsibility Assignment Matrix (RAM) was presented for the preliminary baseline Far Site

Conventional Facilities (FSCF) scope. This included eleven Control Accounts (CAs) and one

Control Account Manager (CAM).

● The project team presented details on lessons learned gathered over the course of implementing the

preliminary baseline.

● One CA was 100% complete at the time of the review (131.01.02.02.02 FSCF Preliminary Design)

and two CAs contain a mixture of LOE and discrete activities below the 15% threshold outlined

within the FRA EVM system description and procedures.

● There were eleven Work Authorization Documents (WADs) provided for the review with updates

through BCR135.

● One CAM was interviewed during the review, responsible for the FSCF preliminary baseline and

CD-3a project scope.

● A preliminary “practice” baseline was established March 2016 for the Far Site Conventional

Facilities (FSCF), WBS 131.01.02.02. The initial baseline was set at $ $317,140,998 in Project

131.01.02.02 (aka “B” project) and does not include completed work under four of the 11 FSCF

control accounts, of $3,644,860, which is in Project 131.01.02.02 A.

● The CD-3a scope is captured as planning packages. These planning packages are broken down into

discrete activities as the contracts are awarded.

● Multiple calendars are used within the project and are assigned at the activity level.

● Funding constraints are on the critical path making the project funding driven until FY19 when the

schedule becomes technically driven.

● The FSCF project has 77 relationships with other projects including links to project office, scope for

each of the caverns, filling the shaft and cryostat, staffing oversight activities and the non-DOE

funded cryostat for the first cavern.

● In the Far Site Conventional Facilities baseline schedule there are 1113 activities, 36 completed, 1 in

progress, 200 with constraints, 85 without predecessors, 29 without successors, 11 out of sequence.



Page 9 of 16

● Project controls works with the CAM to collect, record, and produce monthly performance

information: this includes the Turn-Around Report and Validation Report; Cost and Schedule

Reports; Milestone Reports; Corrective Action Log; and various other miscellaneous reports.

● The project uses the thresholds defined in the Laboratory’s EVM Description for variance analysis.

● The project uses a Corrective Action Log to log and monitor through to completion corrective

actions identified while performing variance analysis.

● CAM reviews project information and assist the project manager in the reporting of project

performance.

● PARS-II Reporting will occur upon completion of the baseline for the CD-3A scope. The project

anticipates a June 2017 upload for the May 2017 reporting period.

● There were 17 changes approved against the FSCF baseline through end of November, with a total

cost impact of $10,788,647, resulting in a total approved budget of $327,929.645.

● Adding completed work (not included in the FSCF baseline) to the end of November FSCF Current

Approved Budget results in the BAC shown on the November 2016 stoplight report of $331,575K

for WBS 131.01.02.02.

● The FSCF CD-3a scope is a subset of WBS 131.01.02.02. The CD-3a scope is contained in 11

unique control accounts and the baseline will be maintained in a separate Cobra project.

● The FSCF CD-3a scope is not being tracked on the FSCF change log. It is, however, being tracked

on the total project log, which has columns to show the BCWS and Schedule Impact at the BCR

summary level for total project, FSCF post Baseline, and CD-3a Scope.

● The FSCF CD-3a scope had an initial baseline of $219,137,742. Changes approved through

November 2016 of $4,988,971, result in an approved budget of $224,126,715. The log shows 18

changes are associated with CD-3a scope

● The HEP Project Status Report – November 2016 for LBNF/DUNE Project includes changes

implemented in October 2016. The Current Approved Value contained in the summary change

status matches the change log for FSCF and FSCF CD-3a, as of end of October 2016.

● There are two change control logs being maintained. The LBNF-DUNE (total project) log and a

FSCF log. The logs are consistent with FRA’s requirements.

● Management Reserve (MR) is held at the LBNF project level and this is noted on the FSCF Change

Control Log. MR will not be partitioned to the CD-3a subproject.

● The FRA Change Control database being developed as the FRA tool is being used by the project and

is expected to finish testing next week.

● The change package for the BCRs contain the reports required by FRA-- BCR Implementation

History Unchanged Validation, BCR Milestone Tier Schedule, BCR Impacted Activities, BCR

Affect on Critical Path, BCR Change Control Account Log --as well as signature information and

Before-After details down to the WP level.

● CR-0125, Update FNAL Rates for 2017, reflected a change in the current period, which Richard

Marcum confirmed had been approved by OPSS.

● The CAM explained that CR-0124 was the movement of scope and budget to an earlier date within

the same fiscal year. The cost impact of $24 does not show up on the WAD BCR Note, because the

WAD Note is in $K.



Page 10 of 16

● Changes are incorporated in a timely manner. The log shows all FSCF changes were implemented

in the same month they were approved. There have been no retroactive changes.

● All changes to the FSCF budget, which were available for review, followed the project’s change

control process and were documented via BCR.

● Current budgets are reconciled to prior budgets, as are start and finish dates, at the control account

level (see BCR Change Control Account Log).

Comments

● Communication between the team, despite being at different geographical locations, appears

effective and well-coordinated.

● The CAM interviewed demonstrated ownership of schedule, cost, and scope information. The CAM

understood responsibilities relating to change control and was able to articulate process steps. The

CAM is knowledgeable about the monthly status reporting requirements, comfortable reviewing

variance documentation, and drafting explanations associated with variances.

● Prior review corrective actions were addressed, if not actually implemented, within the preliminary

baseline. There is one corrective action that still needs to be demonstrated related to long duration

activities and objective information. The project team acknowledged that this needs to be addressed

going forward for the CD-3a scope.

● The project team should be commended for collecting lessons learned during the implementation of

the preliminary baseline. This practice should continue going forward and be shared with other

projects within the laboratory to further learning and growth. The project is encouraged to

incorporate the identified issues into the CD-3a scope.

● Five of the eleven WADs were compliant. The exception being related to one (131.01.02.02.01)

which had not been updated for all November BCRs due to an issue with rounding of delta reports

and five others that had issues with period of performance consistency between the CAP, EV engine,

and schedule. A consistent method of displaying dollar values should be used going forward and an

indicator of change below $K should be considered. Additionally, issues related to the period of

performance inconsistencies should be investigated and corrected.

● Planning Packages use LOE activity types as opposed to task dependent which may cause the project

to show the total float and critical path incorrectly. Additionally, LOE activities use a task dependent

activity type as opposed to LOE which could cause them to appear on the critical path incorrectly or

drive logic within the schedule.

● Discrete activities spanning more than two months must have defined, objective means of measuring

progress. This backup is missing from the schedule within the preliminary baseline and should be

reviewed and updated per the schedule guidelines. This was, also, identified by the project team as a

lesson learned.

● Schedule documentation regarding some lead/lags and constraints is missing within some areas of

the project. These should be reviewed and updated within the schedule.

● Since the completed portion of four control accounts under WBS 131.01.02.02 are budgeted in a

separate project file (131.01.02.02 A), there is inadequate traceability at a control account level



Page 11 of 16

within the preliminary practice FSCF baseline project CA 131.01.02.02. The project needs to resolve

how best to trace approved FSCF budget against the BAC being reported at a WBS or control

account level in any LBNF monthly reports.

● CA 131.01.02.02.02 FSCF Preliminary Design Variance Analysis Report reflects a CPI of 4.44.

This was the result of the project miscalculating remaining/outstanding cost for baseline portion of

this control account. The cost performance index is more in-line with thresholds when the entire

control account is combined with the unbaselined portion of the control account. This can be

confusing to a reviewer when trying to investigate or reconcile performance since the is one CA split

between two baseline files. This also impacts the true variance-at-complete that is being reflected

and can mislead the project team.

● Since the preliminary “practice” baseline and the CD-3a scope changes are being tracked on the total

project change log, the project should see if there is any value added by maintaining a separate FSCF

change log. The items identified as impacting CD-3a should also have impacted the FSCF report.

This should be investigated and corrected as appropriate.

● The initial baseline for FSCF CD-3a scope is expected to be established prior to work starting May

2017. The project will create a separate Cobra file to maintain this baseline. Since there is no MR on

this subset of the project, the Cobra project file - which will be uploaded to PARS-II - should

consider not have the change log turned on. The DOE-FPD will be informed of any changes

processed via the project’s monthly report and, accordingly, will be able to reflect the changed

contingency. How PARS-II data will be uploaded and contingency maintained for CD-3a should be

discussed and resolved with DOE FPD.

● Since CD-3a baseline will only be increased/decreased via use of contingency, the project needs to

resolve with DOE FPD how this will affect BCR approval thresholds.

● On the BCR Implementation History Unchanged Validation report, it would be beneficial to have

the current period highlighted and a note added to reflect authorization of any current period change.

Recommendations

1. Review issues identified with the WADs and correct prior to implementing baseline.

2. Apply lessons learned identified during use of the preliminary baseline within the CD-3a scope.



Page 12 of 16

4.0 Appendices A. Charge

B. Agenda

C. Review Committee Contact List and Writing Assignments



Page 13 of 16

Appendix A

Charge LBNF CD-3a EVMS Implementation Review

January 11-12, 2017



Page 14 of 16



Page 15 of 16

Appendix B

Agenda LBNF CD-3a EVMS Implementation Review

January 11-12, 2017



Page 16 of 16

Appendix C

Review Committee Contact List and Writing Assignments LBNF CD-3a EVMS Implementation Review

January 11-12, 2017

EVMS Assignments

1. Organization – Jennifer Fortner

2. Planning, Scheduling, & Budgeting – Sally Anderson

3. Analysis & Management Reports – Sherese Humphrey

4. Revisions – Fran Leavell

*Accounting was not covered during this review

Charge Question Assignments

Organization – 1, 2, 3, 4, 5, 6, 8

Planning, Scheduling, & Budgeting – 1, 2, 3, 4, 5, 8

Analysis & Management Reports – 1, 2, 3, 4, 5, 6, 8

Revisions – 1, 2, 3, 4, 6, 7, 8