Panel Discussion

Zheng O’Neill PhD,

PE

The University of Alabama

Yeonsook Heo PhD University of Cambridge

Mark Adams Oak Ridge National Lab

Kaiyu Sun Lawrence Berkeley National

Lab

What’s New in Building

Energy Model Calibration?

Contribution

Number

123456

Presentation Date 2017-08-09

Presentation Time 15:30

Covered topics

• Calibrate building model with sensor data (Zheng O’Neill)

• Bayesian calibration (Yeonsook Heo)

• Automated calibration • Optimization-based (Mark Adams)

• Pattern-based (Kaiyu Sun)

Calibration with sensor data

Integrated Model-based Information System

2

Actual Building

Reference Model Predictions

• Energy Consumption

• CO2 Production

• Water Consumption

• Zonal Comfort

Compare

andVisualize

Integrated

Software Environment

Building Energy

Performance Monitoring

Real-Time

Weather

Measurements

• Power Consumption

• Equipment Status

• Water Consumption

• Zonal T, RH

BCVTB

BCVTB: Building Control Virtual Test Bed

Integrate monitoring and simulation models of

building energy use to identify sources of waste and savings potential

Building Model Calibration

Identify uncertain

parameters, perform sampling

Perform numerous

simulations, pre-process output

Calculate full order

meta-model

Create Energy

Model

Calculate reduced-

order meta-model

Perform UA/SA

Perform

Calibration/Optimization

Verify Calibration

0

2000

4000

6000

8000

10000

12000

14000

16000

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

kWh

Plug Electricity

2010 meter data E+ after calibration E+ before calibration

0

5000

10000

15000

20000

25000

30000

35000

40000

1 2 3 4 5 6 7 8 9 10 11 12

kWh

Total Electricity

2010 meter data E+ after calibration E+ before calibration

Error Plug

Electricity

Total

Electricity

MBE 0.02% -2.31%

CV(RMSE) 0.47% 2.80%

• 2063 parameters were varied ± 25% of nominal value • Varied concurrently using a quasi-random sampling

approach • 6500 model realizations • Derivative based sensitivities

• Two outputs (measured data): monthly electricity and plug electricity

• Perform optimization on

2data) - model(

Building Model Calibration Issue • Measured data is significantly far from the baseline model such that changing the

parameters by +/- 25% does not move the output into the range of the measured data

• To alleviate this concern, optimization was performed with constraints on the output variables. Optimal parameters were defined such that the output did not leave an ellipse (in black color) that encompasses the data

1.6 1.7 1.8 1.9 2 2.1 2.2

x 104

0

0.02

0.04

0.06

0.08

0.1

0.12

Total Elec in kWh for January 2010

Pro

babi

lity

Sampled Data

Measured Data

Nominal Model

1.1 1.2 1.3 1.4 1.5 1.6

x 104

1.6

1.7

1.8

1.9

2

2.1

2.2

2.3

2.4x 10

4 March 2010

Plug Electricity [kWh]

Tota

l E

lectr

icity [

kW

h]

Sampled Output

Sensor Data

Baseline Model

Calibrated Meta

Calibrated E+

The meta-model and actual E+ simulation with optimal values has some difference, the meta-model is least accurate at its edges

Bayesian Calibration

New Calibration Methodology

Bayesian Calibration (Kennedy and O’Hagan, 2001)

Bayesian approach updates prior beliefs on true values of uncertain parameters θ in

a computer model given observations

Prior Density Functions for uncertain parameters

Bayesian Calibration

Building Energy Model

Monitored Data (utility bills)

Posterior Distributions

θ1

θn

Bayes’ theorem:

Kennedy and O’Hagan:

Prior Data Likelihood

Parameter Uncertainty

Model Inadequacy

Observation Errors

What is the Value of Bayesian Calibration?

Standard Calibration vs Bayesian Calibration (BC)

Intercept c Indoor Temperature

Infiltration Rate Discharge Coefficient

-3.8 -3.6 -3.4 -3.2 -3 -2.8 -2.6 -2.4 0

0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3

20 20.5 21 21.5 22 22.5 23 23.5

0.6 0.62 0.64 0.66 0.68 0.7 0.72 0.74 0.76

Bayesian process can enhance the reliability of the baseline model

How Does Bayesian Calibration Perform Under High

Uncertainty?

Uncalibrated Model

Calibrated Model

Level 3 0.29 0.04

Level 2 0.32 0.12

Level 1 0.38 0.13

CVRMSE values of model predictions

Is It Necessary to Calibrate All Uncertain Parameters?

Both cases calibrate the top four uncertain parameters:

o Case B: the remaining uncertain parameters are set at base values (mode of uncertainty distribution)

o Case I: the remaining uncertain parameters are fixed at true values (95-quantile of uncertainty distribution)

Optimization-based Automated Calibration

Case Study #1

• Model manually “de-tuned”

• 24 faults

• 63 input changes, +/- 40%

• Goal: Can Autotune recover data it did not use for calibration?

Case Study #1 Results

• Recovery 63 inputs (with +/- 40% ranges)

• 32 of 63 (50%) within 10% of their original value

• 22 (35%) between 10% and 30%

• 9 (14%) above 30%

• Variables important for energy were better

• COP of Chiller:Electric:EIR was off by 0.27%

Case Study #2

• Typical calibration workflow

• Residential Home

• 166 variable changes (35 types), +/- 30%

• Goal: Can Autotune do as well as a human expert calibrator?

Case Study #2 Results

For more information

http://bit.ly/autotune_science

http://bit.ly/autotune_code

Chaudhary, Gaurav, New, Joshua R., Sanyal, Jibonananda, Im, Piljae, O'Neill, Zheng, and Garg, Vishal (2016). "Evaluation of 'Autotune' Calibration Against Manual Calibration of Building Energy Models." In Journal of Applied Energy, volume 182, pp. 115-134, August 2016. [PDF]

Pattern-based Automated Calibration

Pattern-based automated approach • Basic idea => Tune the model according to the patterns of the measured and

simulated data, using experience in modeling and engineering

1 2 3 4 5 6 7 8 9 10 11 12

Months

0

2

4

6

8

10

12

14

Natu

ral G

as U

se (

MW

h)

Simulated Result

Natural Gas Bill

(b)Universal Bias

1 2 3 4 5 6 7 8 9 10 11 12

Months

0

2

4

6

8

10

12

14

Ele

ctr

icity U

se (

MW

h)

Simulated Result

Electricity Bill

(a)Universal Bias

Simulated resultsconsistently lower than electricity use.

Simulated resultsconsistently higher than NG use.

Universal Bias

Seasonal Bias

20

Category Parameters to tune

Internal loads

Occupant density

Lighting power density

Electric equipment power density

Outdoor air flow rate

Infiltration rate

HVAC system

Cooling equipment efficiency

Heating equipment efficiency

Fan efficiency

Cooling set point (schedule)

Heating set point (schedule)

Economizer status

Construction Window U-value

Window SHGC

Schedules

HVAC operation schedule

Lighting schedule

Electric equipment schedule

Case Study

• Single-story office building

• 10,000 ft2

• San Francisco

• Built in 1977

21

22

Tuning Calibration actions Before

tuning

After

tuning

Baseline prior to calibration

Step 1 Decrease lighting power density (W/m2) 21.4 14.9

Step 2 Increase occupant density (person/m2) 0.11 0.14

Step 3 Increase average outdoor air flow per

person (m3/s·person) 0.007 0.008

Step 4 Increase cooling COP 3.1 3.7

…

NMBE NMBE CVRMSE CVRMSE

Electricity Gas Electricity Gas

(%) (%) (%) (%)

… 20.8 -41.2 23.4 50.7

… 6.8 -32.7 10.6 38.9

… 7.6 -9.9 11.3 13.8

… 7.1 -0.2 10.9 14.1

… 4.7 -0.2 8.5 14.1

Baseline

Step1

Step2

Step4

Step3

Electricity Natural Gas

Open Discussion

Questions for panel discussion

• Key impact factors on the calibration accuracy?

• Updates on district/community-level calibration? What are the keys?

• Application of different calibration methods on large-scale data set?

• The application of widely-used sensors (e.g. temperature) data to model calibration?

• Matching monthly or hourly utility bills to Guideline 14 is great, but how do we know the calibrated model matches the real building?

• Is ASHRAE Guideline 14 good enough?

• What metrics define the “best” calibration methodology?

Questions and Discussion

Zheng O’Neill ZONeill@eng.ua.edu

Yeonsook Heo yh305@cam.ac.uk

Mark Adams adamsmb@ornl.gov

Kaiyu Sun ksun@lbl.gov