CEVP ® Cost Estimate Validation Process TEA Conference 2004 Jay Drye, P.E. Assistant State Design Engineer Monica Bielenberg, P.E. CREM Program Manager

CEVP CEVP ®® Cost Estimate Validation ProcessCost Estimate Validation Process

TEA Conference 2004TEA Conference 2004

Jay Drye, P.E.Jay Drye, P.E.Assistant State Design EngineerAssistant State Design Engineer

Monica Bielenberg, P.E.Monica Bielenberg, P.E.CREM Program ManagerCREM Program Manager

Washington StateDepartment of Transportation


Why an Estimate is Not a NumberWhy an Estimate is Not a Number

Estimates are uncertain . . .Ultimate project cost and schedule cannot be known with

certainty during estimating and design Project cost and schedule are functions of many variables

scope and strategyother policytechnical (e.g., associated unit costs and quantities)

They are essentially a snapshot in time


. . . Therefore single number estimates are dangerous“Number” hits the street before estimate is complete or

more often does not include all of the uncertainty inherent to a project


Estimate RefinementEstimate Refinement

Design Level

1% 3% 10% 20% 30%

Cost or Schedule

if many risks occur

if few problems confirmed

Risk Management Objective

Range of Uncertainty

estimate

30

% D

esig

n

100

% D

esi

gn

Co

ns

tru

cti

on

C

om

ple

tio

n

Per

cen

tag

e o

f P

roje

ct C

os

t

Project Development (Time)

Unrecognized Cost

Known butNot Quantified

Known and Quantifiable(can include small uncertainty)

Total Cost

Conservative Estimate

- with Allowance

Contingency

Estimate at any point in time

Pro

jec

t P

lan

&

Co

nc

ept

Uncertainty in Traditional EstimatingUncertainty in Traditional Estimating

All known and unknown risks are equally weighted

Expresses very little knowledge of project risks

Ability to weight known risks according to potential impact to project cost and schedule

Better defines those risks we can identify and refine the amount of unknown risks

Traditional Estimating Risk Assessment

Contingency Risk or Opportunity

Geotech

Enviro

ROW

Unknown

20%40%

30%

10%


Poor assessments of uncertainty lead Poor assessments of uncertainty lead to poor estimates . . .to poor estimates . . .

Impacts of poor cost and schedule estimates include: Cost and schedule over-runs or under-runs Reduction in scope Resource competition Media attention / negative publicity Public mistrust


We have a strong estimating track record:Construction generally completed within 10% of

Engineer’s EstimateCost of change orders during construction 6-7% of bid

priceOverall programs delivered within 3-5% of total

budgeted biennial program

Impacts of Uncertainties: WSDOTImpacts of Uncertainties: WSDOT

However . . . we have our share of ‘black-eyes’ too:

SR 167 1990 $150 million 2000 $972 million


A New Tool Emerged: CEVPA New Tool Emerged: CEVP®®

Evaluate estimate Break down known items, uncertain or “risky” items

(contingencies, allowances, etc) Build back up

Better defined, more accurate numbersInclude uncertain risk/opportunity events

Run simulation model Integrated cost and schedule model

Express costs and durations to complete the project in terms of ranges (distributions)

Prioritize critical risk and opportunity factors to enable more-effective project management


Participants: project team review team

facilitator / elicitorbase assessments risk assessments

technical experts (validate)modeler

CEVPCEVP®® Participants and Process Steps Participants and Process StepsReview Project Scope and Strategy

(Flow Chart and Assumptions)

Review Activity Base Costs,

Durations, and Escalation Rates

Develop Risk Registry

Assess Risk Inputs

Evaluate Uncertainty and Sensitivity in Cost

and Schedule

Report Results Update (optional)

Identify and Evaluate Risk-Management

Strategies and Other Plan Changes (optional)

Develop Cost and

Schedule Uncertainty

Model

Workshops, interviews: preparatory base/risk assessment (1 or 2) risk management / updates


CEVPCEVP®® - Base Cost Determination - Base Cost Determination Determine the “base” costs - the most probable cost that can be expected if the project goes as planned Remove all contingency - i.e. provision for unknowns (representing uncertainty - risk and opportunity)

Consider at the particular stage of the project:

What are our assumptions? Where do they come from? How valid are they, how do we know? What do we know we know? (components, units, prices)

What do we know but can’t quantify? (allowances) What do we know we don’t know? (uncertainty of items) What don’t we know that we don’t know? (uncertainty)


CEVPCEVP® ® - - Develop ProjectDevelop Project Flow ChartFlow Chart Develop the project schedule (“flow chart”) of major

activities required to complete the project

each segment

NEPA /SAC

RODclear

generalpermits/

approvals

pre-RODpermitting

pre-RODROW plan

I-5 IC

Portage BayBridge

Montlake IC

FloatingBridge /

Approaches

PointsSegment

405 IC

finish

pre-RODengr Montlake

Blvd

ROWplan

approval

funding

post-RODpermitting/approvals

post-RODROW plan

post-RODengr

permits

PSE

ROWbid

constructionPhase 1

constructionPhase 2

off-sitemitigation

Bellevue IC

TDM


Identify Risk + Opportunity EventsIdentify Risk + Opportunity Events(Assess Impact + Probability)(Assess Impact + Probability)

Potential Risk or Opportunity

Cost Change

Schedule change

Probability

a) Existing floating Bridge failure, or b) Bridge failure before replacement completed

a) -$170m b) -$35m

a) +24 months b) +24 months

a) 5% b) 3%

Simplify interchange to reduce Right-of-Way taking costs

+$150m -8 months 40%


2. Unacceptable connectivity enhancements

issue: There are no published criteria or detailed accepted science regardingconnectivity for sites like the project site. Because of this, the USFS may not acceptproposed connectivity enhancements at Gold Creek, Price Creek, the connectivitybridge east of MP 62 and at Easton Hill.

impacts: Disagreement about connectivity issues could cause delays in the NEPAprocess and delay the ROD, delays in permitting for the right-of-way, additionalengineering cost and delays in completion of engineering, and additional bridgeconstruction requirements and costs. The Team estimates that this issue could cause a6 month delay in either the NEPA process or the permitting process (but not both).The team also estimates that this issue could cause a construction cost increase of upto $25M.

likelihood: The Team assigned a probability of “possible to likely” for this event, ora 25% chance of occurrence based on input from Paul Wagner, Senior Biologist atWSDOT Office of Environmental Affairs.

mitigation: WSDOT should continue open dialog with the USFS regarding optionsfor connectivity. The project offers significant environmental improvement comparedto the current situation and WSDOT should be aggressive in pursuing credit for out-of-kind mitigation in other areas.

Risk Event - DetailRisk Event - Detail

Evaluate UncertaintyEvaluate Uncertainty

Uncertainty in Activity “Base” CostsUncertainty in Activity “Base” Durations

Activity A

Activity C

Activity B

Start End

Uncertainty inTotal Project Costand Duration

$

T

7 56

121110

8 4

21

9 3

7 56

121110

8 4

21

9 3

Risk Events (likelihood of occurrence, and likelihood for cost and durationchanges if the event occurs)

Event Y

Event X


Combine base costs, risk and opportunity events, with probabilities, to create potential ranges of cost & schedule

00.020.040.060.08

0.10.120.140.16

15

25

16

00

16

75

17

50

18

25

19

00

19

75

20

50

21

25

22

00

Cost ($M)

Pro

ba

bil

ity

R ank R elative C ontrib ution

to R isk C ost1 R isk o r O pp ortu nity E vent

1 23% O ngoin g ETC Gov ernance & Staff Issu es 2 18% O &M S ubsidy R isk 3 9% C ontracting Process 4 7% A dditional Parking R eq uired 5 7% other risk item s 6 7% U rban D esign Risk 7 6% U tility Relocation Issues 8 5% O ther Scope R isk 9 3% P ower System s Cost Un certainty

10 3% Foun dation Design R isk all others 12%

Cost Risk Contribution By ItemR ank

R elative C ontrib ution to R isk C ost1 R isk o r O pp ortu nity E vent

1 23% O ngoin g ETC Gov ernance & Staff Issu es 2 18% O &M S ubsidy R isk 3 9% C ontracting Process 4 7% A dditional Parking R eq uired 5 7% other risk item s 6 7% U rban D esign Risk 7 6% U tility Relocation Issues 8 5% O ther Scope R isk 9 3% P ower System s Cost Un certainty

10 3% Foun dation Design R isk all others 12%

Cost Risk Contribution By Item

CEVPCEVP®® ResultsResults


Schedule comparison after Risk Schedule comparison after Risk Management was initiatedManagement was initiated

66

75

84

93

10

2

11

1

12

0

12

9

13

80

0.05

0.1

0.15

0.2

0.25P

rob

ab

ilit

y

Total Project Duration (months after Jan/03)

Jul-02 Mar-03


What does it take to do CEVPWhat does it take to do CEVP®®??

A knowledgeable/committed owner (who wants to know the “potential cost”)

A well-shaped project estimate Available/involved project team members Sufficient independent subject matter experts Skilled risk and cost elicitors (debiasing) Risk modeling - technology and experience Time / available funding


W a s h i n g t o n S t a t eD e p a r t m e n t o f T r a n s p o r t a t i o n

T o o l f o r P r o j e c t D e l i v e r yT o o l f o r P r o j e c t D e l i v e r y

I m p r o v e d R i s k M a n a g e m e n t I m p r o v e d E s t i m a t e / B u d g e t M a n a g e m e n t B e t t e r d e c i s i o n s e a r l y i n t h e

d e v e l o p m e n t p r o c e s s S u c c e s s f u l t o o l t o c o m m u n i c a t e t o t h e

p u b l i c T i m e a n d R e s o u r c e C o m m i t m e n t


““CEVPCEVP®® produced very useful results” produced very useful results”

“The Transportation Department developed its new numbers through a new process called “cost estimate validation” or CEVP, which features another layer of review by outside experts…The agency’s Urban Corridors Administrator characterized it as an effort to deal more openly and honestly with risks and uncertainties. ”

Seattle Times, June 2002

“Giving citizens a range of costs, including full disclosure of the variables, “is not only politically smart, but it’s common sense”…”

Seattle Post-Intelligencer, June 2002

“…transportation department effort to plan more accurately and manage money more effectively…So give DOT some credit for those intimidating estimates. They should show us that it’s way past time to pass a funding package and get to work. Delay will only increase the cost.”

Spokesman-Review June 2002


CEVPCEVP®®

CEVP® has been registered by WSDOT to recognize their sponsorship of its development and to ensure that the term is not loosely applied in other settings to cost review procedures that contain less than all the tools and controls that have been incorporated into the process, as used at WSDOT.


Questions ? ? ?Questions ? ? ?

Documents

CEVP ® Cost Estimate Validation Process TEA Conference 2004 Jay Drye, P.E. Assistant State Design Engineer Monica Bielenberg, P.E. CREM Program Manager