CEVP CEVP ®® Cost Estimate Validation ProcessCost Estimate Validation Process
TEA Conference 2004TEA Conference 2004
Jay Drye, P.E.Jay Drye, P.E.Assistant State Design EngineerAssistant State Design Engineer
Monica Bielenberg, P.E.Monica Bielenberg, P.E.CREM Program ManagerCREM Program Manager
Washington StateDepartment of Transportation
Washington StateDepartment of Transportation
Why an Estimate is Not a NumberWhy an Estimate is Not a Number
Estimates are uncertain . . .Ultimate project cost and schedule cannot be known with
certainty during estimating and design Project cost and schedule are functions of many variables
scope and strategyother policytechnical (e.g., associated unit costs and quantities)
They are essentially a snapshot in time
Washington StateDepartment of Transportation
. . . Therefore single number estimates are dangerous“Number” hits the street before estimate is complete or
more often does not include all of the uncertainty inherent to a project
Washington StateDepartment of Transportation
Estimate RefinementEstimate Refinement
Design Level
1% 3% 10% 20% 30%
Cost or Schedule
if many risks occur
if few problems confirmed
Risk Management Objective
Range of Uncertainty
estimate
30
% D
esig
n
100
% D
esi
gn
Co
ns
tru
cti
on
C
om
ple
tio
n
Per
cen
tag
e o
f P
roje
ct C
os
t
Project Development (Time)
Unrecognized Cost
Known butNot Quantified
Known and Quantifiable(can include small uncertainty)
Total Cost
Conservative Estimate
- with Allowance
Contingency
Estimate at any point in time
Pro
jec
t P
lan
&
Co
nc
ept
Uncertainty in Traditional EstimatingUncertainty in Traditional Estimating
All known and unknown risks are equally weighted
Expresses very little knowledge of project risks
Ability to weight known risks according to potential impact to project cost and schedule
Better defines those risks we can identify and refine the amount of unknown risks
Traditional Estimating Risk Assessment
Contingency Risk or Opportunity
Geotech
Enviro
ROW
Unknown
20%40%
30%
10%
Washington StateDepartment of Transportation
Poor assessments of uncertainty lead Poor assessments of uncertainty lead to poor estimates . . .to poor estimates . . .
Impacts of poor cost and schedule estimates include: Cost and schedule over-runs or under-runs Reduction in scope Resource competition Media attention / negative publicity Public mistrust
Washington StateDepartment of Transportation
We have a strong estimating track record:Construction generally completed within 10% of
Engineer’s EstimateCost of change orders during construction 6-7% of bid
priceOverall programs delivered within 3-5% of total
budgeted biennial program
Impacts of Uncertainties: WSDOTImpacts of Uncertainties: WSDOT
However . . . we have our share of ‘black-eyes’ too:
SR 167 1990 $150 million 2000 $972 million
Washington StateDepartment of Transportation
A New Tool Emerged: CEVPA New Tool Emerged: CEVP®®
Evaluate estimate Break down known items, uncertain or “risky” items
(contingencies, allowances, etc) Build back up
Better defined, more accurate numbersInclude uncertain risk/opportunity events
Run simulation model Integrated cost and schedule model
Express costs and durations to complete the project in terms of ranges (distributions)
Prioritize critical risk and opportunity factors to enable more-effective project management
Washington StateDepartment of Transportation
Participants: project team review team
facilitator / elicitorbase assessments risk assessments
technical experts (validate)modeler
CEVPCEVP®® Participants and Process Steps Participants and Process StepsReview Project Scope and Strategy
(Flow Chart and Assumptions)
Review Activity Base Costs,
Durations, and Escalation Rates
Develop Risk Registry
Assess Risk Inputs
Evaluate Uncertainty and Sensitivity in Cost
and Schedule
Report Results Update (optional)
Identify and Evaluate Risk-Management
Strategies and Other Plan Changes (optional)
Develop Cost and
Schedule Uncertainty
Model
Workshops, interviews: preparatory base/risk assessment (1 or 2) risk management / updates
Washington StateDepartment of Transportation
CEVPCEVP®® - Base Cost Determination - Base Cost Determination Determine the “base” costs - the most probable cost that can be expected if the project goes as planned Remove all contingency - i.e. provision for unknowns (representing uncertainty - risk and opportunity)
Consider at the particular stage of the project:
What are our assumptions? Where do they come from? How valid are they, how do we know? What do we know we know? (components, units, prices)
What do we know but can’t quantify? (allowances) What do we know we don’t know? (uncertainty of items) What don’t we know that we don’t know? (uncertainty)
Washington StateDepartment of Transportation
CEVPCEVP® ® - - Develop ProjectDevelop Project Flow ChartFlow Chart Develop the project schedule (“flow chart”) of major
activities required to complete the project
each segment
NEPA /SAC
RODclear
generalpermits/
approvals
pre-RODpermitting
pre-RODROW plan
I-5 IC
Portage BayBridge
Montlake IC
FloatingBridge /
Approaches
PointsSegment
405 IC
finish
pre-RODengr Montlake
Blvd
ROWplan
approval
funding
post-RODpermitting/approvals
post-RODROW plan
post-RODengr
permits
PSE
ROWbid
constructionPhase 1
constructionPhase 2
off-sitemitigation
Bellevue IC
TDM
Washington StateDepartment of Transportation
Identify Risk + Opportunity EventsIdentify Risk + Opportunity Events(Assess Impact + Probability)(Assess Impact + Probability)
Potential Risk or Opportunity
Cost Change
Schedule change
Probability
a) Existing floating Bridge failure, or b) Bridge failure before replacement completed
a) -$170m b) -$35m
a) +24 months b) +24 months
a) 5% b) 3%
Simplify interchange to reduce Right-of-Way taking costs
+$150m -8 months 40%
Washington StateDepartment of Transportation
2. Unacceptable connectivity enhancements
issue: There are no published criteria or detailed accepted science regardingconnectivity for sites like the project site. Because of this, the USFS may not acceptproposed connectivity enhancements at Gold Creek, Price Creek, the connectivitybridge east of MP 62 and at Easton Hill.
impacts: Disagreement about connectivity issues could cause delays in the NEPAprocess and delay the ROD, delays in permitting for the right-of-way, additionalengineering cost and delays in completion of engineering, and additional bridgeconstruction requirements and costs. The Team estimates that this issue could cause a6 month delay in either the NEPA process or the permitting process (but not both).The team also estimates that this issue could cause a construction cost increase of upto $25M.
likelihood: The Team assigned a probability of “possible to likely” for this event, ora 25% chance of occurrence based on input from Paul Wagner, Senior Biologist atWSDOT Office of Environmental Affairs.
mitigation: WSDOT should continue open dialog with the USFS regarding optionsfor connectivity. The project offers significant environmental improvement comparedto the current situation and WSDOT should be aggressive in pursuing credit for out-of-kind mitigation in other areas.
Risk Event - DetailRisk Event - Detail
Evaluate UncertaintyEvaluate Uncertainty
Uncertainty in Activity “Base” CostsUncertainty in Activity “Base” Durations
Activity A
Activity C
Activity B
Start End
Uncertainty inTotal Project Costand Duration
$
T
7 56
121110
8 4
21
9 3
7 56
121110
8 4
21
9 3
Risk Events (likelihood of occurrence, and likelihood for cost and durationchanges if the event occurs)
Event Y
Event X
Washington StateDepartment of Transportation
Combine base costs, risk and opportunity events, with probabilities, to create potential ranges of cost & schedule
00.020.040.060.08
0.10.120.140.16
15
25
16
00
16
75
17
50
18
25
19
00
19
75
20
50
21
25
22
00
Cost ($M)
Pro
ba
bil
ity
R ank R elative C ontrib ution
to R isk C ost1 R isk o r O pp ortu nity E vent
1 23% O ngoin g ETC Gov ernance & Staff Issu es 2 18% O &M S ubsidy R isk 3 9% C ontracting Process 4 7% A dditional Parking R eq uired 5 7% other risk item s 6 7% U rban D esign Risk 7 6% U tility Relocation Issues 8 5% O ther Scope R isk 9 3% P ower System s Cost Un certainty
10 3% Foun dation Design R isk all others 12%
Cost Risk Contribution By ItemR ank
R elative C ontrib ution to R isk C ost1 R isk o r O pp ortu nity E vent
1 23% O ngoin g ETC Gov ernance & Staff Issu es 2 18% O &M S ubsidy R isk 3 9% C ontracting Process 4 7% A dditional Parking R eq uired 5 7% other risk item s 6 7% U rban D esign Risk 7 6% U tility Relocation Issues 8 5% O ther Scope R isk 9 3% P ower System s Cost Un certainty
10 3% Foun dation Design R isk all others 12%
Cost Risk Contribution By Item
CEVPCEVP®® ResultsResults
Washington StateDepartment of Transportation
Schedule comparison after Risk Schedule comparison after Risk Management was initiatedManagement was initiated
66
75
84
93
10
2
11
1
12
0
12
9
13
80
0.05
0.1
0.15
0.2
0.25P
rob
ab
ilit
y
Total Project Duration (months after Jan/03)
Jul-02 Mar-03
Washington StateDepartment of Transportation
What does it take to do CEVPWhat does it take to do CEVP®®??
A knowledgeable/committed owner (who wants to know the “potential cost”)
A well-shaped project estimate Available/involved project team members Sufficient independent subject matter experts Skilled risk and cost elicitors (debiasing) Risk modeling - technology and experience Time / available funding
Washington StateDepartment of Transportation
W a s h i n g t o n S t a t eD e p a r t m e n t o f T r a n s p o r t a t i o n
T o o l f o r P r o j e c t D e l i v e r yT o o l f o r P r o j e c t D e l i v e r y
I m p r o v e d R i s k M a n a g e m e n t I m p r o v e d E s t i m a t e / B u d g e t M a n a g e m e n t B e t t e r d e c i s i o n s e a r l y i n t h e
d e v e l o p m e n t p r o c e s s S u c c e s s f u l t o o l t o c o m m u n i c a t e t o t h e
p u b l i c T i m e a n d R e s o u r c e C o m m i t m e n t
Washington StateDepartment of Transportation
““CEVPCEVP®® produced very useful results” produced very useful results”
“The Transportation Department developed its new numbers through a new process called “cost estimate validation” or CEVP, which features another layer of review by outside experts…The agency’s Urban Corridors Administrator characterized it as an effort to deal more openly and honestly with risks and uncertainties. ”
Seattle Times, June 2002
“Giving citizens a range of costs, including full disclosure of the variables, “is not only politically smart, but it’s common sense”…”
Seattle Post-Intelligencer, June 2002
“…transportation department effort to plan more accurately and manage money more effectively…So give DOT some credit for those intimidating estimates. They should show us that it’s way past time to pass a funding package and get to work. Delay will only increase the cost.”
Spokesman-Review June 2002
Washington StateDepartment of Transportation
CEVPCEVP®®
CEVP® has been registered by WSDOT to recognize their sponsorship of its development and to ensure that the term is not loosely applied in other settings to cost review procedures that contain less than all the tools and controls that have been incorporated into the process, as used at WSDOT.