1 NORTHWEST ENERGY EFFICIENCY ALLIANCE DHP UES Key Issues & Decisions Ecotope, Inc. May 21, 2013

1 NORTHWEST ENERGY EFFICIENCY ALLIANCE

DHP UES Key Issues & Decisions

Ecotope, Inc.May 21, 2013

2

Agenda

Introduction Presentation Objectives Measure Description

Research Structure

Summary of Savings

Summary of Key Issues

Issues & StrategiesQ&AMotions

3

Introduction

4

Presentation Objectives

Review research and findings for DHP pilot evaluation (2008-2013)

Develop a common foundation of understanding for assessing DHP UES DHP performance

Technical vs. behavioral and/or program design considerations

Key issues for DHP UES

Make several fundamental decisions in advance of full UES workbook presentation

5

Measure Description

Measure Classification and Properties

Market Sector ResidentialMarket Segment Single-familyMeasure Category Seeking to convert provisional to

proven.Measure Description Install a ductless heat pump in

the main living area of an existing zonal-electrically heated house.

6

DHP Research Structure

UES Workbook

Billing Analysis ~4,000 Participants

Market Progress Evaluation~300 Participants

Field Monitoring95 Participants

Lab Testing 2 units

7

Saving Distribution, 3,887 kWh/yr

Electric SavingsSupplemental FuelsTakeback

8

Wide Range of DHP Savings

Metered sample Careful screening to ensure all-electric heat Measured heat contribution from the output of the DHP Analyzed pre-installation bills to establish base heating use

Billing Analysis Screened based on poor VBDD fit Screened based on intake questionnaire

Determined supplemental fuel savings impact Determined takeback from improved heating signature

Lower R2 for pre-installation indicates an electric heat “takeback”, R2 measure fit, improves with more electric heat and with consistent thermostat

settings When pre-installation is lower screen and supplemental fuels flag is set to zero,

the screen is set assuming that this would indicate electric heat takeback

Substantial agreement with results of metered sample

9

Savings Vary by Screening Approach

Savings by Participant Group Savings (kWh/yr)Screened bills: R2>.45, Space Heat>0 Mean EB n

All 2081 81 3390No Supplemental Fuels Used 2719 90 2295

No Takeback (R2) 2976 101 1705

Metered Results: Measured Savings from Metered Sub-Sample

Total Savings (COP) 3887 375 65

Billing Analysis 3049 251 93

Electric Sav-ings

Supplemental Fuels

Takeback

10

Summary of Issues

Issue 1: Which savings estimation approach: Calibrated Engineering (SEEM) or Statistical/Metastatistical (Billing & Metering)?

Issue 2: How to determine electric kWh savings and supplemental fuel benefit?

Issue 3. How/whether to include “comfort” in the measure savings and/or TRC calculation?

Issue 4. Should we combine the savings, or disaggregate by climate zone?

11

Issue 1: Which savings estimation approach: Calibrated Engineering (SEEM) or Statistical/Metastatistical (Billing &

Metering)?

12

UES Estimation Procedures Statistical

DHP pilot population only represents the pilot project participants and is not a representative sample of the region

Pilot evaluation uses a large billing analysis with an intake questionnaire to provide participant information

The billing analysis provides an impact evaluation framework but is not suited to developing the determinants of consumption

More detailed information available on the performance of the DHP from lab and field monitoring

Meta-Statistical Both the billing analysis and the engineering analysis provide a direct measure of the savings

potential Comparable studies in size and scope have not been attempted elsewhere Small engineering studies in this region can be used to expand this analysis but are not focused

on this measure and the electric zonal heat population targeted

Calibrated Engineering Integrated research design focused on the performance characteristics of the DHP

Detailed lab testing of system performance Submetering provide confirmation of lab test and determinants of consumption across a number of climates

and participants (95) Testing and metering results allow the development of a DHP model within the SEEM simulation

Analysis shows the structure of the DHP performance in real homes Calibration of SEEM model can be quite accurate and allow the results of the research to be

generalized for climates and house types

13

Tiered Research Approach Shows Agreement Among Various Performance Measurements

Lab Measurements

Field Performance Measurements

Lab supports field findings

DHP Performance Simulation

Field Energy Use Measurements

Annual Energy Use Simulation:

Final Savings Estimates

Billing Data

Field measurements & billing data agree

Lab & field data combine to develop equipment performance models

Primary Measurements: Calibrated Models:

Billing & field data used to calibrate simulation

14

Measured Performance

Lab data forms the truth set for performance measurements Provides critical insight into

equipment operation including Maximum & minimum capacity Low power cut off / cycling limit Defrost operation Indoor unit fan power System standby power

Lab data verifies the accuracy of the field measurements

Lab & field data combined to develop equipment performance models

15

Performance Curve Development

3 curves constructed from field data HSPF Levels: 12, 10.6, 8.2 Curves are the modal COP values for a given

temperature bin They incorporate all transient effects including defrosting and

part load cycling (previous box plots were steady-state only)

0 10 20 30 40 50 60 70 800

0.51

1.52

2.53

3.54

4.55

DHP1

10619 12095 13170

13183 Predicted

Outdoor Temp (F)

CO

P

0 10 20 30 40 50 60 70 800

1

2

3

4

5

6

7DHP2

10564 10626 11279

11446 Predicted 12411

Outdoor Temp (F)

CO

P

0 10 20 30 40 50 60 70 800

1

2

3

4

5

6

7DHP3

11936 12321

12322 Predicted

Outdoor Temp (F)

CO

P

16

Multi-Zones with a Single-Zone Model?

SEEM is a single zone heating simulation DHPs and ER zonal systems are, by definition, multi-

zone Simulation implementation is to determine the

fraction of the entire house heated by the DHP at every hour (DHPƒ). Then, fill in the remaining heating requirement with resistance heat.

Field measurements show DHPƒ depends strongly on outdoor temperature and house heat loss rate

17

DHPƒ Field Observations

Fraction of total metered heating provided by the ductless heat pump

DHPƒ = (DHPheat) / (DHPheat + ERheat)

18

Multi-Zones with a Single-Zone Model

Logistic function used for DHPƒ curve fit within SEEM:

OAT is outside temperatureUA is house heat loss c1 and c2 are constantsfrom a regression fit.

19

Measured and Modeled Heating Energy

Pre / Post DHP

Source

Heating Energy Use (kWh/yr)

N

Mean SD

Pre

Billing Data 9347 3892 91

SEEM 65.8°F Set point 9369 4111 91

SEEM 67.3°F Set point 10384 4409 91

Post

Metered Data 6484 3894 91

SEEM 65.8°F Set point 5881 3018 91

SEEM 67.3°F Set point 6485 3234 91

SEEM can model both the base case houses (ER heat only) and the post case houses (mix of DHP and ER heating) Per other calibration efforts, house models constructed from detailed audit

data and the SEEM t-stat setting changed to match metered energy

20

Agreement: Metered Sites to Billing Sites

95 metered sites are a subpopulation of the 4,000 billing analysis sites Metered sites used no supplemental fuel and were continuously occupied year round

When we examine the energy savings the billing population would

have were it to use no supplemental fuel and be fully occupied, we

see the same savings as from the meters and simulation.

Screened billing analysis savings: ~2,975 kWh/yr for 1706 houses agrees

well with calibrated modeled savings from previous slide of 2,865 kWh/yr

for 91 sites.

Link between metered and billing populations shows we can

expect SEEM to correctly simulate energy use under a variety of

conditions.

21

SEEM is Calibrated to Model Energy Use in Houses with DHPs

Sound Engineering & Statistical Analysis: Lab and field measurements show us how the equipment performs apart from the occupant.

Reliable Data & Calibration: We can use the simulation with confidence because it is calibrated and validated against field data.

Flexibility: The simulation of the equipment allows us to model various program designs and occupant behaviors giving the possibility to move beyond this pilot project population.

22

Issue 2: How to determine electric kWh savings and supplemental fuel benefit?

23

Distribution of Total DHP Savings


24

Options for Supplemental Heat

Option 1: Electric kWh savings are based on results of the billing/metering studies (first-year); supplemental fuel is quantified as found in the evaluation, and monetized. Analysis:

Uses billing analysis results from the pilot program Uses savings that are consistent with supplemental fuel use across the region

Pros: Reduces uncertainty in savings estimate

Cons: Potentially underestimates long-term savings Increases risk that changing fuel preferences will increase electric baseline Reduces savings estimate for homes without supplemental fuel use May require differential savings based on supplemental fuel use

Option 2: Electric kWh savings are sum of first year savings and supplemental fuel savings (long term = no wood use) Analysis:

Uses the results of detailed metering as the basis for savings calculation Pros:

Hedge against changing fuel preferences Cons:

May over estimate electric savings

Option 3: Somewhere in between Option 1 and 2; the percentage of long-term wood use is based on ________. Pros:

Distributes uncertainty across short and long-term Cons:

May underestimate savings

25

Pilot Project Eligibility and Intake Screens

Electric resistance (ER) heating must be permanently installed and serve as the primary heating system for the home. (Screen for presence of ER; no screen to establish ER as primary heat, meaning no screen for wood/propane as de facto primary heat)

The consumer must have occupied the home for one year prior. (In addition, the consumer should expect to occupy the home for the next two years.). (No screen for length of past/future occupancy)

The consumer must allow their local utility to make their billing histories available. (Included in terms of intake form)

The consumer must agree to participate in the project, project activities, and project evaluation. (Included in terms of intake form)

Participating homes cannot: be new construction (No screen for new construction) have natural gas service to the home. (Screen for gas service)

26

Savings and Supplemental Fuels

Supplemental Fuels Documented in installation questionnaire

Largely wood heat Some contractors did not ask about supplemental fuel

use (especially in some western climate areas) Dominated by rural participants

Screened out of the metering sample (not quantified)

Supplemental fuels more significant in some areas Rural areas of western climates. Most eastern climates (especially Montana)

27

Supplemental Fuel Use by Pilot Population*

*Based on data in Baylon, D., P. Storm, and D. Robison. 2013. Ductless Heat Pump Impact & Process Evaluation: Billing Analysis Report, Northwest Energy Efficiency Alliance. Portland OR.

Regional avg. = .35

28

Supplemental Fuel Impact

Supplemental fuel impacts can dominate savings estimates Electric savings impacted by supplemental fuels

Generally Wood, some propane and other fuels Differential wood use in eastern clusters Uncertainty in use of supplemental heating

Reduction in wood use implied by negative bill “savings” Screening suggests about 2000 kWh/yr savings reduction

between homes with and without supplemental fuels CDA regression used to assess the size of the supplemental

fuel and other takeback effects Supplemental fuel savings impact 400 kWh/yr across the entire pilot

program Savings impact 700 kWh in eastern utilities and 350 kWh in western utilities

Supplemental fuel is a occupant choice that does not effect the “primary” electric heating system. Electric heating system remains in place

29

Willamette (n=547)

Puget Sound (n=247)

Coastal (n=95)

Inland Empire (n=65)

Boise/Twin (n=29)

Eastern Idaho (n=30)

Tri-Cities (n=14)

Western Montana (n=68)

Total (n=1095)

-400

0

-300

0

-200

0

-100

0

0 1000

2000

3000

4000

Savings Estimate (kWh/year; 90% confidence interval)

Clu

ster

Space Heating Savings: Supplemental Fuels*

*Based on screened billing analysis in Baylon, D., P. Storm, and D. Robison. 2013. Ductless Heat Pump Impact & Process Evaluation: Billing Analysis Report, Northwest Energy Efficiency Alliance. Portland OR.

30

Space Heating Savings: No Supplemental Fuels*

Willamette (n=1454)

Puget Sound (n=454)

Coastal (n=138)


Boise/Twin (n=63)


Tri-Cities (n=37)


Total (n=2295)

-400

0

-300

0

-200

0

-100

0

0 1000

2000

3000

4000


Clu

ster


31

How to monetize supplemental fuels?

If supplemental fuel benefits are included in the analysis (Option 1 and 3), how should they be monetized?

Option A: Value of wood taken as the avoided cost of wood fuel: Attempt to value wood directly.

Option B: Value of wood taken as the avoided cost of electricity: Use wholesale electricity prices.

Option C: Value of wood taken as the cost of electricity: Use retail electricity prices.

32

Issue 3: How/whether to include “comfort” in the measure savings and/or TRC calculation?

33

Distribution of Total DHP Savings


34

How/whether to include “comfort” in the measure savings and/or TRC calculation?

Option 1: Do not include in analysis at all.

Option 2: Include in TRC analysis; convert to $’s (benefit).

Option 3: Add to kWh savings.

35

Temperature and Comfort (MPER)

Nearly all (97%) of respondents reported that their home was more comfortable (91%) or equally comfortable (6%) than it was prior to installing the DHP.

Only two respondents indicated that the home was less comfortable.

Respondents gave a variety of ways in which their comfort was improved by the DHP.

36

Thermostat Settings and Setback (MPER)

Pre and post-DHP avg. heating temp. reported at 69-70° F

Pre-DHP, 69% reported setback when leaving the house or at night; 23% said they never setback.

Post-DHP, 42% reported setback when leaving the house or at night; 35% said they never setback.

57% said they were heating the same amount pre and post-DHP, 40% said they were heating it more. The most common reason given for heating the area more was that the new heat was cheaper to operate (61%).

37

Savings with Temperature Setting Offsets

The DHP offers an improved thermostat control and a reduction in heating energy. Reduced savings inferred from the temperature

observed in the metering. Regression suggests a 125 kWh savings reduction per ºF

SEEM calibration suggested a temperature adjustment. About 2 ºF increase in overall thermostat developed in the

calibration Billing analysis CDA showed other savings

offsets after supplemental fuels considered. Thermostat setting one possible source of takeback

38

Detailed interview: Metered Sample

Temperature changes from interviews for metered sample 20% said they increased temperature by

about 3ºF on average. 9% said they decreased temperature by

about 3ºF on average. Relation between reported t-stat setting and

measured temperature suggests a higher setting than reported by about 1ºF.

39

Inferred Temperature Impacts

Temperature impacts not directly observed Metering only included post installation conditions No questions on heat setting

Indicator variable developed to track homes with better heat signatures Increased R2 in post installation period Assigned indicator to the 40% largest increases in

R2save Most screened out with supplemental fuels Some screened out with pre installation R2 screening

Screened savings “takeback” summaries based on this variable

40

Space Heating Savings: No Takeback*

Willamette (n=1113)

Puget Sound (n=339)

Coastal (n=66)


Boise/Twin (n=39)


Tri-Cities (n=31)


Total (n=1706)

-400

0

-300

0

-200

0

-100

0

0 1000

2000

3000

4000


Clu

ster


41

Space Heating Savings: Temperature Increase Indicator*

Willamette (n=342)

Puget Sound (n=115)

Coastal (n=72)


Boise/Twin (n=24)

Eastern Idaho (n=1)

Tri-Cities (n=6)


Total (n=590)

-400

0

-300

0

-200

0

-100

0

0 1000

2000

3000

4000


Clu

ster


42

“Takeback” Decisions

T-stat increases account for savings takeback SEEM predicts about 400 kWh/ºF Temperature take back up to 800 kWh/yr in metered

sample Bill screening suggests 650 kWh/yr per home with this

indicator CDA predicts 500 kWh/yr for homes with this indicator

Should these takebacks be included as part of the DHP savings benefits? Option 1: Do not include in analysis at all. Option 2: Include in TRC analysis; convert to $’s (benefit). Option 3: Add to kWh savings.

43

Issue 4: Should we combine the savings, or disaggregate by climate zone?

44

Use Similar DHP Savings Total: All climates

Similar installation standards in all climates Displacement model anticipates optimum output and cost

effectiveness in all climates. Installation standardized at 1-1.5 tons throughout the region Occupant use similar across region Supplemental fuel most significant determinant of savings in all climates.

Evaluation of COP by climate shows similar response in metered group Savings dominated by swing seasons in cold climates Savings available year round in warm climates Cooling not a significant offset in any climate

Average space heat fraction differs between western and eastern climates but absolute saving similar Use of wood heating different between west and east so uniform

grid savings not expected

45

DHP Total Savings Results*

Willamette (n=18)

Puget Sound (n=19)


Boise/Twin (n=8)

Eastern Idaho (n=9)

Total (n=65)

0 1000

2000

3000

4000

5000

6000

Savings Estimate (kWh/year; 90% confidence interval)C

lust

er

*Based on metered heat output measurements in Baylon, D., L. Larson, P. Storm, and K. Geraghty. 2012. Ductless Heat Pump Impact & Process Evaluation: Field Metering Report, Northwest Energy Efficiency Alliance. Portland OR.

46

Installations Similar Across Climates*

ClusterCapacity (tons)

One Indoor Unit All

Willamette 1.4 1.7

Puget Sound 1.3 1.6

Coastal 1.2 1.4

Inland Empire 1.7 1.9

Boise/Twin 1.3 1.8

Eastern Idaho 1.3 1.6

Tri-Cities 1.3 1.6

W. Montana 1.3 1.5

Total 1.4 1.6

*See Baylon, D., L. Larson, P. Storm, and K. Geraghty. 2012. Ductless Heat Pump Impact & Process Evaluation: Billing Analysis Report, Northwest Energy Efficiency Alliance. Portland OR.

47

Wood Heat Varies by Occupant not Climate*

*Based on conditional demand analysis (CDA) data in Baylon, D., P. Storm, and D. Robison. 2013. Ductless Heat Pump Impact & Process Evaluation: Billing Analysis Report, Northwest Energy Efficiency Alliance. Portland OR.

Regional avg. = 393

48

DHP measure should be uniform, based on one regional specification and savings.

Total savings very uniform across climates.

Supplemental fuels should be handled separately in assigning “grid” savings.

With common displacement spec common savings should be anticipated.

Climate distinction an unnecessary complication.

49

Questions & Answers

50

Motions

51

Motion 1: Savings Estimation Approach

“I ____ move that the calibrated engineering approach be used to estimate savings for DHP measures and programs.”

52

Motion 2: Approach to Supplemental Fuels

“I _______ move to include supplemental heat in the DHP savings estimate analysis using the following option:Option 1: First year savings approach. Electric kWh savings are based on results of the billing/metering studies (first-year); supplemental fuel savings is quantified as found in the evaluation, and monetized.

Option 2: Long-term savings approach. Electric kWh savings are sum of first year savings and supplemental fuel (long term = no wood use)

Option 3: Hybrid approach. Somewhere in between Option 1 and 2; the percentage of long-term wood use is based on ______.

53

Motion 3: Approach to Takeback

“I _______ move to include or exclude takeback offsets in final savings estimates.”

54

Motion 4: Uniform DHP Measure

“I _______ move to use a single savings estimate for DHP total savings across all climate zones.”

55

Additional Slides

56

Savings Vary by Screening Approach

Savings by Participant Group Savings (kWh/yr)

Screened bills: R2>.45, Space Heat>0 Mean EB n

All 2081 81 3390

Supplemental Fuels Used 747 144 1095

No Supplemental Fuels Used 2719 90 2295

Takeback based on R2 1971 181 590

No Takeback 2976 101 1705

Metered Results: Measured Savings from Metered Sub-Sample

Total Savings (COP) 3887 375 65

Billing Analysis 3049 251 93

57

Performance Curves Compared

First 2 are for the units

tested in the lab and cover

a high HSPF range.

Third curve, based solely

on field data, developed

because we needed a

lower performing model.

The COP begins to bend

down at warmer

temperatures because of

equipment cycling.

58

CDA Billing Analysis Segmentation

Assess the overall savings and the space heating savings from the DHP installations in the pilot project (n=3620)

Determine the impact of takeback effects on observed savings using CDA

Underlying savings rate (c1) similar to meters

Supplemental fuels offset (c2) consistent except MT

Constant term (C) shows impact of other occupancy takebacks

59

CDA Regression Results

Climate Zone Segment

Parameter

nc1 c2 C

Western 0.487 -973 -768 3122

Eastern 0.223 -1,152 -300 375

W. Montana 0.249 -1,683 -416 123

All 0.434 -1,110 -561 3620

SHsaved=c1SHpre+c2SuppFuel+C

Climate Zone Segment

Parameter

n

c1 c2 C

Est. EB Est. EB Est. EB

Western 0.479 0.016 -1078 131 -676 140 3,122

Eastern* 0.219 0.046 -1220 456 -226 519 375

W. Montana* 0.241 0.096 -1761 1263 -275 1545 123

All 0.426 0.015 -1208 129 -466 139 3,620

Climate Zone

Segment

Parameter

n

c1 c2 C

Est. EB Est. EB Est. EB

Western 0.479 0.016 -1078 131 -676 140 3,122

Eastern* 0.219 0.046 -1220 456 -226 519 375W. Montana* 0.241 0.096 -1761 1263 -275 1545 123

All 0.426 0.015 -1208 129 -466 139 3,620

60

CDA Savings Estimates

CDA savings estimates similar to the screened billing analysis results Supplemental fuel decrements the total

savings by about 1200 kWh/yr in those homes

Other “Takeback” captured by constant term Regression did not include R2 indicator Screening results suggest about a third of the

constant effect is explained by this indicator Overall savings represents the comparable

screening to the metered sample

61

Segmented Regression Results

62

CDA Predicted Space Heating Savings

Willamette (n=2086)

Puget Sound (n=752)

Coastal (n=285)


Boise/Twin (n=96)


Tri-Cities (n=55)


Total (n=3621)

-400

0

-300

0

-200

0

-100

0

0 1000

2000

3000

4000


Clu

ster

63

The CDA comparison

Overall savings from the CDA equivalent to screened billing analysis Based only on space heat coefficient (c1) Comparable to Metered Sample billing savings

Overall “takeback” from all sources equal to a third of predicted savings Supplemental fuels account for 40% of that reduction Temperature “takeback” accounts for about 25% of that

reduction The balance is the result of occupancy or other behavior

How should these effects be included in the cost/effectiveness analysis?

64

Savings and C/B Recommendations

Supplemental fuel taken as the value of the offset electric heat

Temperature and occupancy effects should be ignored

Total savings should be used for TRC cost effectiveness Based on the c1 coefficient c1= .48 for western climates

.24 for eastern climates Grid Savings would account for observed

takeback especially supplemental fuels

65

Modeled and Measured COP

SEEM Modeled

Metered

SEEM Modeled

Metered

SEEM Modeled

Metered

SEEM Modeled

Metered

SEEM Modeled

Metered

SEEM Modeled

Metered

Pu

ge

t S

ou

nd

(n

=2

0)

Bo

ise

/Tw

in (

n=

8)

Ave

rag

e/T

ota

l (n

=6

9)

2 2.5 3 3.5 4

COP (90% confidence interval)

Documents

1 NORTHWEST ENERGY EFFICIENCY ALLIANCE DHP UES Key Issues & Decisions Ecotope, Inc. May 21, 2013