Civil Engineering @ University of Toronto Management

Performance Measurement in Asset Management—Financial indicators

Civil Engineering @ University of Toronto

Tamer E. El-Diraby, PhD., PEng.

Professor, University of Toronto

Progress in engineers-dominated R&D

Need progress in the financial prediction

Predictions in AM are complex/chaotic

The Era of no-model prediction

New policy directions

• Re-define performance

• Invest in digital twinning

• Consider project options

• Consider policy scenarios

|K | e | y p | o | i | n | t | s |

Just in case you did not notice

The Agenda

• Framing the issues

• Synthesis of financial/economic indicators

Intermission: Q&A

• Reengineering the decision making process

• The role of AI

Framing the problem

What is the question? That is the problem.

What are you funding?

• Asset rehabilitation?

• The management of the assets?

• The capacity for managing asset, optimally?

Mohseni, H., Setunge, S., Zhang, G., & Edirisinghe, R. (2012). Deterioration prediction of community buildings in Australia. International Journal of the Constructed Environment.

• Is the agency (AM process) worth funding?

• Can the agency justify ROI in the broader

socio-economic context?

An overview: Decision making framework

Define performance

Physical deterioration

Levels of service

Operational efficiency

Resiliency Capacity

Rec

ogn

ize

life

cycl

e

Collect data

Predict performance deterioration

Predict/estimate costs

Assess risks/priority

Assess criticality

Scen

ario

s/p

olic

y o

bje

ctiv

es

Maintain/ enhance

Consistent funding or as

needed?

Source of funding?

Define ROI

Boundary conditions

Safety/codeUser

satisfactionEnergy/

obsolesce Climate change

Service delivery

Model/simulate/iterate

Technical backlog Functional backlog Capacity backlog

Technical index/model Functional index/model Capacity index/model

SustainmentMaintain LOS

Preserve asset

ImprovementEnhance LOSModernize

DevelopmentExpand LOS

Provide new asst

OperationsExpand LOS

Optimize costs

Financial Predictions

Financial Prediction for Building Maintenance

Financial prediction in buildings

Age-dominated(THE CURVE)

Simple models

Rules of thumb

Based on life cycleValue-based

Condition oriented

Econometric

Financial perdition models in buildings (rules of thumb)

2-4% rule

• First suggested by NRC (USA) in 1990

• Based on personal experience: 1% is too low

and no organization can afford 5%

• The exact % depend on age, construction

quality, level of use, quality of maintenance

system, climate

• Good for large mixed inventory, not correct for

each building, not good for long-term forecast

NRC. (1991). Pay Now or Pay Later: Controlling Cost of Ownership from Design throughout the Service Life of Public Buildings, National Research Council, National Academy Press, Washington, D.C. Rush, S. C. (1990). “Facilities as a Capital Asset.” Facilities Stewardship in the 1990s, Washington, D.C., 1-18. Tolk, J. N. (2007). Predicting required maintenance and repair funding based on standard facility data elements (Doctoral dissertation, Texas Tech University).

Kraft (1950)

• M&R budget = CRV x MCF where MCF is a maintenance cost factor based on construction type

• M&R budget = % x CRV of facility where the % is the choice of the decision maker.

• M&R budget = facility square footage x cost factor where the cost factor is derived from historical data.

• M&R budget = last period budget x cost factor increase based on inflation.

• M&R budget = % PRV + % DM backlog + new construction – demolition.

• Contrast/triangulate several approaches

Financial perdition models in buildings (Simple formulae)

Tolk, J. N. (2007). Predicting required maintenance and repair funding based on standard facility data elements (Doctoral dissertation, Texas Tech University).

Barco (1994)• Funding doesn’t rely on facility condition

• It relies on a backlog list of unfunded projects.

• Value-based models depend on the timeliness and quality of the backlog data.

The Coast Guard Methodology (Lofgren, et al., 1999) • Combines replacement cost, square foot, and incremental budgeting using the (BOMA) standard.

• Recurring maintenance is based on cost per square foot. Non-recurring work is an incremental

budget, plus 1.5 – 2 %.



Monterecy (1985) • Rather than facility attributes that directly affect M&R costs, looked at the

eight reasons most often cited for having a deferred maintenance backlog.

• The predictor variables are high energy cost, age, poor construction quality,

demographic changes, lack of maintenance staff, compliance with federally

mandated improvements, and lack of facility planning.

• Only facility age, facility planning, and construction quality are significant



Dergis & Sherman (1981)

• Based on construction costs, current year building value,

• Renewal should cost less than replacement, recognizes older facilities require more funds

• Reflects the effect of renewal already accomplished

• Annual M&R funding = 2/3 BV x BA/1275, where

2/3 = building renewal constant

BV = current building value

BA = building age as corrected for partial or total renewal

1275 = age weighting constant for 50 years using sum of years digit



Kaiser (1989)

• condition assessment, which is considered the most comprehensive approach; and

life-cycle analysis of actual backlog when data does not exist.

• Empirical studies have resulted in ranges of 1.5-3% PRV as an appropriate level.

• “A one time elimination of current renewal and replacement priorities does not solve

the problem.”



Turner (1996)

• A formula based on three assumptions (scenarios):

• funding is allocated during each two-year legislative session,

• logical factors such as size or age of facilities should be used, and

• the method must accommodate phased-in funding increases.

Financial perdition models in buildings (Models)


Neathammer & Neely (1991) (unit cost)

• Benchmarked construction costing

• Developed a series of databases to predict maintenance resources under several funding

scenarios:

• Generated a 120-year service life projection of possibilities.

• Annual cost for labor equipment for each trade or shop for a known floor area, usage, age,

labor and equipment per hour.

• Their goal was to determine the high cost components and tasks that are cost drivers in

building maintenance and use them to develop an accurate resource prediction.

Financial perdition models in buildings (Process)


The BUILDER Model (parametric estimation)

• An M&R estimate and decision

• An inventory, inspection, condition assessment, deterioration modeling, condition prediction, and M&R planning.

• A hierarchy based on 12 facility systems and 12 specialties.

• Systems, components, and subcomponents are inventoried to establish hierarchy, then condition assessments are

based on a statistical sample.

• The model produces a condition index, which is used on component deterioration to determine repair costs.

• The model can be used to predict both current and anticipated facility condition for the portfolio, and can be used for

“what- if” scenarios for funding situations.

Financial perdition models in buildings (Process)


Durango (2002) (Pavement)

• Using closed-loop and open-loop-feedback control formulations, compared them to a single model, static

representation of deterioration (model triangulation)

• Extended open-loop optimal feedback control formulation to a network level problem.

• Generation of a set of condition-dependent policies that account for heterogeneities in the network.

• An agency will make a policy decision, then observe deterioration after the application of the policy.

• They cannot judge how much did the policy itself impact deterioration (the lab vs real world data).

• Deterministic models assume the condition can be predicted with certainty for a period in the planning horizon.

Financial perdition models in buildings (advanced models)


Why is it hard

Bortolini, R., & Forcada, N. (2020). A probabilistic performance evaluation for buildings and constructed assets. Building Research & Information, 48(8), 838-855.

The FCI

The FCI

Cecconi, F. R., Moretti, N., & Dejaco, M. C. (2019). Measuring the performance of assets: a review of the Facility Condition Index. International Journal of Strategic Property Management, 23(3), 187.

Variations on FCI

The Maintenance Efficiency Indicator (MEI)

• Used for assessment of efficiency of maintenance expenditure.

• Not always possible to cover the total amount of DM

• A target CI (calculated as the CI, namely 1-FCI) must be defined.

• Replacement is not considered in the calculation of this metric.


Variations on FCI

Replacement Efficiency Indicator

• Trend for substitution of systems at the asset level.

• REI below 100% means that amount spent for replacement is

not enough.

• Components are not replaced each year, thus REI is not a

continuous function.

• Calculation at the single component level may not be significant.


Shortcomings of FCI

• FCI is mainly led by the value of the denominator. For

instance, two analogous low cost (deferred) maintenance

interventions on two components characterized by very

different CRV values determine substantial differences in the

FCI calculation.

• the FCI will be more critical for the component featured by

the lower CRV value. Nevertheless, that simple intervention

could be highly critical for the component featured by the

highest replacement cost. Therefore, a simple FCI

calculation could not be representative of the criticality of

the maintenance operation

the ratio of the FCI of two components changes according to

variations in the ratio between the deferred maintenance

costs (DM ratio) and the current replacement values (CRV

ratio) of the components. for a given DM ratio the FCI ratio

increases with a linear trend as well as the CRV ratio. For a

given CRV ratio the FCI ratio decreases following a

hyperbolic trend with the DM ratio increasing. Cecconi, F. R., Moretti, N., & Dejaco, M. C. (2019). Measuring the performance of assets: a review of the Facility Condition Index. International Journal of Strategic Property Management, 23(3), 187.

• The lab vs real world dilemma: Deterioration is a function of maintenance policy/quality

• The context problem: deterioration is a function of boundary conditions (climate, level of use, etc.)

• Data size and quality could be more important than the formula itself

• Facility attributes vs backlog reasons

• Feedback: what maintenance took place; what was the effect

• Funding scenarios and what-if scenarios

• Funding consistency beyond a year

• Partial options must be considered

• Network-level analysis

Key points

• The curve is deterministic and is now outdated in the research community

• The alternative is an advanced model that is encapsulated in a software that is driven by local data

• It should consider simulating what-if and possibilities

• It should include an optimization mechanism

• The implementation of the model is to be done through a “process” presented as a software

• The features of the software depend on the extent of policy making maturity/sophistication

• The quality of the software results are based on data quality, size and consistent collection

• You must consider scenarios of funding

• The scenarios must consider funding levels, types, consistency, funding sources

• Funding can be allocated based on policy objectives: assure certain goals and to promote others, when possible

Preliminary conclusions

Deterioration curves for pavement (an appendix)

Deterministic Statistical Artificial intelligence

Linear Markov chain Case-based reasoning

Exponential Ordinal regression Fuzzy set theory

Logarithmic Linear discriminant analysis Neural networks

Polynomial Gamma process —

Power Gaussian process —

Performance vs Deterioration

What is Performance?

Lai, J. H., & Man, C. S. (2018). Performance indicators for facilities operation and maintenance (Part 2): Shortlisting through a focus group study. Facilities.

Physical

Environmental

TechnicalFinancial

Why not measure performance?


Find a short, feasible list


(1) (P1) thermal comfort;(2) (P4) indoor air quality;(3) (P5) percentage users dissatisfied;(4) (F0) ratio of total O&M cost to building income;(5) (F4) actual costs within budgeted costs;(6) (F13) O&M cost per building area;(7) (T1) work request response rate;(8) (T12) number of completed work orders per staff;(9) (T13) area maintained per maintenance staff;(10) (T18) backlog size;(11) (T25) failure/breakdown frequency;(12) (T28a) availability of fire services system;(13) (T28b) availability of lift;(14) (E1) energy use index;(15) (E3) greenhouse gas emission per building area;(16) (H1) number of accidents per year; and(17) (H6) number of lost work days per year.

Understand Performance


Alright3

Intermission: Q&A

A decision-making framework

An overview: Decision making framework

Define performance

Physical deterioration

Levels of service

Operational efficiency

Resiliency Capacity

Rec

ogn

ize

life

cycl

e

Collect data

Predict performance deterioration

Predict/estimate costs

Assess risks/priority

Assess criticality

Scen

ario

s/p

olic

y o

bje

ctiv

es

Maintain/ enhance

Consistent funding or as

needed?

Source of funding?

Define ROI

Boundary conditions

Safety/codeEngage

communityEnergy/

obsolesce Climate change

Service delivery

Model/simulate/iterate

Technical backlog Functional backlog Capacity backlog

Technical index/model Functional index/model Capacity index/model

SustainmentMaintain LOS

Preserve asset

ImprovementEnhance LOSModernize

DevelopmentExpand LOS

Provide new asst

OperationsExpand LOS

Optimize costs

Decision chains

Options

ROICost

Performance

Project scope

Criticality

Simulation

Data quality

Unit level

Ministry level

MOI level

Use

rs/C

omm

unity

out

reac

h

Trea

sury

Boa

rd/F

inan

cial

Indu

stry

Budget allocation

Funding sources

Scenarios

Optimization

Decision making system

Virtualize, engage

Simulate, triangulate, optimize

Dat

a re

po

sito

ry

Data Management system

Real-world data & Test cases

Common data model

BIM

Data collection standards

IoT standards

Data quality standards

Data governance

Machine learning system for performance prediction

Curve or model-based performance prediction

Simple formula

EnergyLOSSafety capacityClimate

Project options

Machine learning system for performance prediction Machine learning system for

performance prediction Policy scenarios

Project optionsProject options

Project optionsProject options

Cost estimates & ROI

Refine models

Examine policies

Analytics/R&D

Refine models

Learning

Update model

Macro trends

AI & the climate: the no-model approach

Developing the curve (Pavement Example, City of Oshawa); No Traffic data

Buildings

Remember, there is, actually, a band of curves—not just one?

Could the top box reparents the natural deterioration and the next represent deterioration 2nd and 3rd cycles?

To get a single curve, we should average the three boxes

The deterioration of do-nothing?

Possible band of curves (torturing the limited data)?

Data analytics, Climate-based deterioration prediction

Triangulate between models

Model #1 Model #2

(7 level)

Model #3

(7 level)

Model #4

Model #1A(5 levels)

Model #1B(7 levels)

Model #4A(5 levels)

Model #4B(7 levels)

Initial PCI value X X X X X XAge of road X X X XAnnual average daily traffic X X

Years since last remedial action X X X X

Type of last remedial action X X XFunctional class X Type of pavement X X X XGranular base equivalence X XNumber of freeze-thaw cycles X X X X XAnnual freeze index X X X XAv. daily max. temperature X Av. daily min. temperature X Total annual precipitation X X

Accuracy 72% ±4.7% 67% ±4.7%67% ±3%

67% ±3%

78% +/- 5 75% +/- 5

Accuracy (data cleaned) 73% ± 4.4 87% +/- 5.81% 82% +/- 5.81%

Prediction map

Climate change impacts

Deterioration of 44 roads in Ontario with ongoing climate patterns

Climate change impacts

Deterioration of 44 roads in Ontario with changed climate: slower deterioration

Policy Scenarios (frequency of work, type of work, PCI, budget)

• Types of work

• A, B, C, D

• Frequency of work

• 2, 4, 6, 12

• Minimum PCI: 80%

• PCI: TxDOT formula

• Costs: historical records

• Transition: Markov chain

• Applicable work

condition Applicable1 A, B, C*, D*

2 A, B, C*, D*

3 B**, C, D4 C, D

Segment condition Applicable work type/level

1 A, B, C*, D*

2 A, B, C*, D*

3 B**, C, D

4 C, D

Policy scenarios: cost, frequency of work, type of work, PCI

Clustering by climate and road function

• PCI deterioration (in different

climates) is based on TxDOT formula

• Road function

• A: arterial

• L: local

• Climate

• C: Cold climate

• R: relatively warm and humid

• W: warm & dry

Clustering by climate and road function

Thanks, eh!

The gap/centrality of “cost data”

• Ontario has a

good/established

culture in condition

data collection

• Cost data collection is

• Not on par

• Not linked to

condition data

Wooldridge, S. C. (2001). Balancing capital and condition: An emerging approach to facility investment strategy (Doctoral dissertation, Massachusetts Institute of Technology).

Analysis means—the influence of engineers

Evaluate/predict performance

Evaluate/predict CRV

Estimate costs

Decision analysis and

policy comparisons

Decision making

Ad hoc/ manual/ expert deliberations

Indices Statistical analysis Simulation

AI

Optimization

Decision authority: fund by objective?

Wooldridge, S. C. (2001). Balancing capital and condition: An emerging approach to facility investment strategy (Doctoral dissertation, Massachusetts Institute of Technology).

What is performance

Lavy, S., Garcia, J. A., & Dixit, M. K. (2010). Establishment of KPIs for facility performance measurement: review of literature. Facilities.

Measuring performance indicators


Nature of deterioration


Simplification factors

Mohseni, H., Setunge, S., Zhang, G., & Edirisinghe, R. (2012). Deterioration prediction of community buildings in Australia. International Journal of the Constructed Environment.

Types of maintenance


Deliverable 4: LOS

• In some cases, LOS could be the driver for capacity change.

• However, in the majority of cases, enhancement LOS is considered as added outcome for rehabilitation

work.

• As such, LOS is typically incorporated in the decision-making as a prioritizing feature.

• Reporting on the enhancement of LOS due to rehabilitation or replacement of assets is a key consideration

in assessing the ROI of asset management decisions.

• Integrated decision-making systems provide means to consider LOS as an independent factor. i.e. they base

the decision-making formula on a holistic performance index that could incorporate safety, energy, resilience

and LOS.

Deliverable 4: Climate action

• Considering climate change in asset management decision making spans two fundamental modes:

• Enhancing asset resilience to climate change. This can include, for example:

• Replacing existing assets with more advanced/resilient assets--for example, better roofing systems.

• Adding green infrastructure systems to protect assets from climate change impacts--for example, adding

erosion control infrastructure to mitigate the impacts of severe storms on structures.

• Reconfiguring the asset to contribute to climate action. This can include, for example:

• Replacing assets with ones that have lower energy consumption and/or carbon footprint.

• Reducing the need to use facilities. This is a recent trend that is increasingly being considered, especially after

COVID-19. In this regard, reliance on e-business systems will reduce the need for expansion or frequent uses of

physical facilities which result in reduced energy consumption to operate facilities.

• There are four fundamental challenges to incorporating climate change in asset management

• What climate: the decision-maker has to adopt a model for estimating the changes in climate. What is the expected

change in temperature, precipitation, storm frequency?

• What to include: should the socio-cultural changes associated with climate change be considered? For example, expected

that human migration?

• In what way: understand the relationship between climate hazards and the deterioration of infrastructure. This is difficult

due to four reasons:

• Which climate change hazard has an impact on which asset

• The complexity of hazard interaction. For example, the temperature rise and precipitation.

• Modeling the relationship: in what way and to what extent does a climate hazard hasten or reduce deterioration.

• How much: what is the delta rehabilitation cost that is associated with the positive or negative impact of change in

climate hazards.

Deliverable 4: Climate change

Deliverable 4: Climate change

• Make an educated assumption on the percentage change or speed of deterioration

• Develop objective formulas for measuring the impacts of climate change on rates of deterioration

• Within integrated decision making (ones that go beyond using a simple deterioration curve), consider, model

and simulate several scenarios.

Documents

Civil Engineering @ University of Toronto Management