origin-sl.michigan.govorigin-sl.michigan.gov/documents/.../Appendix...7.docx · Web viewThe Home Energy Report (HER) program, implemented by the investor-owned utilities in Michigan

XX-XX: Behavioral Modification: Home Energy Reports

Residential (Single and Multi-Family)

Electric and Natural Gas

Issued: 10 / 1 7 / 20 1 7

Prepared by: Navigant Consulting, Inc.

Version: 1.5

Michigan Residential Measures 1 Version 1.5

Document Revision History

Version # Revision Date Description of Revisions and Affected Sections Edited By

1.0 April 1, 2017 1. Annual application for the Nth year measure. Specifically, introduces two new measures:

a. 6th year 7k-9k kWh Annual Usageb. 6th year 9k-11k kWh Annual Usage

2. Updated energy and peak demand savings values for the 5th year measure using a complete 12 months of data

DTE Energy

1.1 May 31, 2017 1. A calibration study resulted in proposed updates to savings for existing measures, as well as the introduction of new usage bands for existing measures. The list of measures are identified in Section 1

2. Section 2 was updated to provide the background and context for the calibration study and proposed revisions to the measures

3. Updated energy savings values based on results of calibration study in Section 4

4. Sections 6, 7 and 8 present the methodology and results of the calibration study, including results of the regression analysis and method for accounting for potential overlapping savings.

Navigant Consulting, Inc.

1.2 July 31, 2017 1. Year 6 values were finalized and reported in Sections 2 and 4.3.

DTE Energy

1.3 August 15, 2017

1. 90% confidence interval reported in section 7.1 Navigant Consulting, Inc.

1.4 September 21, 2017

1. Removed the word “Preliminary” from the Year 6 Gas Measure Table in Section 4.3. Year 6 values were finalized on July 31 but the word “preliminary” was not removed from the table.


1.5 October 17, 2017

1. Updated 5-7 MWh PY1 value from 0.48% to 0.70% in Electric Measure Table in Section 4.3. Initial savings value was reported in error and was based on only one cohort. The updated savings value reflects savings estimate associated with two eligible cohorts.

2. Swapped 7-9 MWh and 9-11 MWh PY4 values in Electric Measure Table in Section 4.3 as these values were mistakenly transposed when developing the whitepaper.

3. Error identified in example calculation in section 7.2. Multiplication of whole number rather than percent (e.g., 1.26 rather than 0.0126).


Michigan Residential Measures i Version 1.5

EO Collaborative Review and Decision

Date Description of Collaborative Decision Lead

Michigan Residential Measures ii Version 1.5

Table of Contents

1. New Measure and Update to Existing Measure...........................................................................................1

2. Purpose and Measure Background...............................................................................................................2

3. Measure Description........................................................................................................................................2

4. Measure Specification & Savings Summary.................................................................................................3

5. Baseline and Proposed Improvement Description.......................................................................................4

6. Methodology and Assumptions......................................................................................................................5

7. Estimated Energy Savings over Baseline.....................................................................................................7

8. Potential Savings Overlap.............................................................................................................................10

9. Coincidence Factor........................................................................................................................................13

10. Measure Life...................................................................................................................................................13

11. Measure Cost..................................................................................................................................................13

12. Any Recurring Cost........................................................................................................................................13

13. Relevant Codes and Standards...................................................................................................................13

14. Ongoing EM&V, Research, and Calibration Planning...............................................................................13

15. Sources of Information...................................................................................................................................14

16. Attachments....................................................................................................................................................14

Michigan Residential Measures iii Version 1.5

1. New Measure and Update to Existing Measure

1. Updates to Existing Measures

This whitepaper proposes updates to existing electric and gas measures based on the results of a calibration study.

Measure Code Electric/Gas Measure

N-RE-MS-010122-C-XX-XX-XX-XX-02 Elec/Gas Behavior Modification: Home Energy Reports, 9k-11k kWh Annual Usage (1st year)

N-RE-MS-010123-C-XX-XX-XX-XX-02 Elec/Gas Behavior Modification: Home Energy Reports, 7k-9k kWh Annual Usage (1st year)

N-RE-MS-010124-C-XX-XX-XX-XX-03 Elec/Gas Behavior Modification: Home Energy Reports, 9k-11k kWh Annual Usage (2nd year)

N-RE-MS-010125-C-XX-XX-XX-XX-03 Elec/Gas Behavior Modification: Home Energy Reports, 7k-9k kWh Annual Usage (2nd year)

N-RE-MS-010172-C-XX-XX-XX-XX-02 Elec/Gas Behavior Modification: Home Energy Reports, 9k-11k kWh Annual Usage (3rd year)

N-RE-MS-010173-C-XX-XX-XX-XX-02 Elec/Gas Behavior Modification: Home Energy Reports, 7k-9k kWh Annual Usage (3rd year)

N-RE-MS-010174-C-XX-XX-XX-XX-01 Elec/Gas Behavior Modification: Home Energy Reports, 9k-11k kWh Annual Usage (4th year)



2. New Measures

This whitepaper proposes two electric usage bands (5-7k and >11k kWh annual usage) in addition to the two usage bands that currently exist (7k-9k and 9k-11k kWh annual usage). In addition, this whitepaper proposes introducing two usage bands to gas savings (900-1200 and >1200 therms annual usage), replacing the current gas savings.

Electric/Gas Measure

Elec Behavior Modification: Home Energy Reports, 5k-7k kWh Annual Usage (1st year)

Elec Behavior Modification: Home Energy Reports, 5k-7k kWh Annual Usage (2nd year)

Elec Behavior Modification: Home Energy Reports, >11k kWh Annual Usage (1st year)

Elec Behavior Modification: Home Energy Reports, >11k kWh Annual Usage (2nd year)

Gas Behavior Modification: Home Energy Reports, 900-1200 Therms Annual Usage (1st year)

Gas Behavior Modification: Home Energy Reports, 900-1200 Therms Annual Usage (2nd year)

Gas Behavior Modification: Home Energy Reports, 900-1200 Therms Annual Usage (3rd year)

Gas Behavior Modification: Home Energy Reports, 900-1200 Therms Annual Usage (4th year)

Gas Behavior Modification: Home Energy Reports, 900-1200 Therms Annual Usage (5th year)


Electric/Gas Measure

Gas Behavior Modification: Home Energy Reports, >1200 Therms Annual Usage (1st year)

Gas Behavior Modification: Home Energy Reports, >1200 Therms Annual Usage (2nd year)

Gas Behavior Modification: Home Energy Reports, >1200 Therms Annual Usage (3rd year)

Gas Behavior Modification: Home Energy Reports, >1200 Therms Annual Usage (4th year)

Additionally, this whitepaper proposes updates to existing electric and gas measures based on additional energy savings data that has become available since version 1.0 of this whitepaper was published on April 1, 2017.

Electric/Gas MeasureElec/Gas Behavior Modification: Home Energy Reports, 9k-11k kWh Annual Usage

(6th year) Elec/Gas Behavior Modification: Home Energy Reports, 7k-9k kWh Annual Usage

(6th year)

2. Purpose and Measure Background

This whitepaper reports the results of the calibration of the Behavior Modification: Home Energy Report measure. Compared to the values included in the 2017 Michigan Energy Measures Database, this calibration:

1. Updates electric energy and gas savings values for all usage bands and program years for which data are available. This includes:

- Electric energy savings for year 1, 2, 3, 4 and 5 values for the 7,000 to 9,000 kWh annual usage band

- Electric energy savings for year 1, 2, 3, and 4 values for the 9,000 to 11,000 kWh annual usage band

2. Introduces electric energy savings values for two additional usage bands and program years for which data are available. This includes:

- Electric energy savings for year 1 and 2 values for the 5,000 to 7,000 kWh annual usage band

- Electric energy savings for year 1 and 2 values for the greater than 11,000 kWh annual usage band

3. Replaces all gas savings values with gas savings values in two usage bands for all program years for which data are available. This includes:

- Gas savings for year 1, 2, 3, 4 and 5 values for the 900 to 1,200 therm annual usage band- Gas savings for year 1, 2, 3 and 4 values for the greater than 1,200 therm annual usage

band

4. Updates electric energy and gas savings values for program year 6 based on additional energy savings data that has become available since version 1.0 of this whitepaper was published on April 1, 2017.

- Electric energy savings for year 6 values for 7,000 to 9,000 kWh annual usage band- Electric energy savings for year 6 values for 9,000 to 11,000 kWh annual usage band


3. Measure Description

The Home Energy Reports program provides residential households accurate and timely information on their energy consumption through a variety of communication methods to change the consumers’ energy usage behavior.

The Home Energy Reports program is organized around two concepts. First, motivate consumers through normative messaging to change their energy-use behavior. Personalized neighbor comparisons based on the size of the home, location, and heating fuel type—among other criteria—give households a motivational benchmark for their energy usage. Second, provide them with salient, personalized advice to capitalize on this motivation to use less energy and save money.

Home Energy Reports are delivered through direct mail and are often supplemented with digital communications such as email, the web, mobile phones, and social networks. This platform approach ensures that all households can access this information in the most effective way.

Applicable Building Type(s)

Applicable Sub-

categories

Applicable Fuel Type(s) Affected End-Use Load(s)

Electricity Natural Gas

Space Heating

Space Cooling

Water Heating Lighting Appliance Process Other

Residential n/a

4. Measure Specification & Savings Summary

4.1 Measure Specification

Behavior Modification Reports are a tool that can be deployed to residential customers to drive a behavioral change to reduce home energy consumption. Key information includes:

1. Comparison of the customer’s home energy use to neighbors’ energy usage

2. An energy consumption grade

3. Comparison of the current period’s energy usage with a past period’s use and comments on increased or decreased energy utilization

a. Typically, this compares the current month or quarter with the same month or quarter from the prior year.

4. Suggested actions the customer can take to improve energy efficiency including some low or no cost ideas, as well as higher impact ideas that may require capital expenses

In addition, the program must have a simple opt-out process.

4.2 Requirements for Application

The Behavior Modification Report is restricted to single family residences.

4.3 Savings Summary


Electric Measures

Program Year Usage Band Electric Savings per Customer

Coincident Peak Demand Savings

1st Year > 11 kWh Annual Usage 1.20% 1.80%1st Year 9k-11k kWh Annual Usage 1.08% 1.62%1st Year 7k-9k kWh Annual Usage 0.84% 1.26%1st Year 5k-7k kWh Annual Usage 0.70% 1.05%2nd Year > 11 kWh Annual Usage 1.78% 2.67%2nd Year 9k-11k kWh Annual Usage 1.52% 2.28%2nd Year 7k-9k kWh Annual Usage 1.50% 2.25%2nd Year 5k-7k kWh Annual Usage 0.77% 1.16%3rd Year > 11 kWh Annual Usage n/a n/a3rd Year 9k-11k kWh Annual Usage 1.77% 2.66%3rd Year 7k-9k kWh Annual Usage 1.82% 2.73%3rd Year 5k-7k kWh Annual Usage n/a n/a4th Year > 11 kWh Annual Usage n/a n/a4th Year 9k-11k kWh Annual Usage 1.25% 1.88%4th Year 7k-9k kWh Annual Usage 2.27% 3.41%4th Year 5k-7k kWh Annual Usage n/a n/a5th Year > 11 kWh Annual Usage n/a n/a5th Year 9k-11k kWh Annual Usage 2.18%^ 3.27%^5th Year 7k-9k kWh Annual Usage 2.01% 3.02%5th Year 5k-7k kWh Annual Usage n/a n/a6th Year > 11 kWh Annual Usage n/a n/a6th Year 9k-11k kWh Annual Usage 2.12%* 3.18%*6th Year 7k-9k kWh Annual Usage 1.70%* 2.55%*6th Year 5k-7k kWh Annual Usage n/a n/a

Note: Energy savings quoted as percentage of baseline energy consumption. ^ Value represents the existing value in the 2017 MEMD due to insufficient data for calibration.* The addition of these two measures reflects new data available on the savings performance of the HER program in its 6th year of continuous operation at DTE. The savings data for these two measures are based on actual savings for the period May 2016 –April 2017.

Gas Measures

Program Year Usage Band Gas Savings per Customer

1st Year 900-1200 Therms Annual Usage 0.34%1st Year >1200 Therms Annual Usage 0.43%2nd Year 900-1200 Therms Annual Usage 0.53%2nd Year >1200 Therms Annual Usage 0.60%3rd Year 900-1200 Therms Annual Usage 0.91%3rd Year >1200 Therms Annual Usage 0.57%4th Year 900-1200 Therms Annual Usage 0.86%4th Year >1200 Therms Annual Usage 0.66%5th Year 900-1200 Therms Annual Usage 0.66%5th Year >1200 Therms Annual Usage 1.09%^6th Year 900-1200 Therms Annual Usage 0.73%*6th Year >1200 Therms Annual Usage 0.73%*

Note: Energy savings quoted as percentage of baseline energy consumption.^ Value represents the existing value in the 2017 MEMD due to insufficient data for calibration.* The addition of these two measures reflects new data available on the savings performance of the HER program in its 6th year of continuous operation at DTE. The savings data for these two measures are based on actual savings for the period May 2016 –April 2017.

5. Baseline and Proposed Improvement Description


5.1 Baseline Measure, Practice, or Technology

The baseline measure is a control group customer that does not receive energy usage feedback and benchmarking from a paper or email Home Energy Report.

5.2 Improved Measure, Practice, or Technology

The behavior modification report is an improved measure in that it provides customer-specific energy usage feedback, benchmarking and tips to identify potential savings opportunities, thus altering participant behavior and energy consumption.

6. Methodology and Assumptions

The Home Energy Report (HER) program, implemented by the investor-owned utilities in Michigan (DTE Energy and Consumers Energy), delivers Behavior Modification Reports and employs a randomized controlled trial (RCT) design that allows for the accurate and unbiased measurement of program impacts. In an RCT design, eligible households are randomly assigned to either a treatment group that receives Behavior Modification Reports or a control group that does not receive Behavior Modification Reports. A sound randomization leads to treatment and control groups that are statistically equivalent along observed and unobserved characteristics with the sole difference being that the treatment group receives Behavior Modification Reports, and the control group does not. Therefore, any difference in energy usage observed between the treatment and control group can be attributed to the Behavior Modification Reports. The RCT program design is summarized in Figure 1.

Figure 1: Experimental Setup for a Randomized Controlled Trial

Experimental design is considered the “gold standard” of program evaluation and a RCT is recognized as an appropriate method for evaluation in the United States Department of Energy’s Uniform Methods Project1 and in an evaluation protocol report authored by the State and Local Energy Efficiency (SEE) Action Network in 2012. Experimental designs eliminate self-selection bias, wherein customers who choose to join a program are different from those who choose not to join, making the savings estimates from experimental designs known to be unbiased.

1 The Uniform Methods Project is coordinated by the National Renewable Energy Laboratory (NREL).Michigan Residential Measures 5 Version 1.5

The energy savings resulting from the behavior modification report are calculated using a regression-based ex post evaluation of the HER program, comparing participant energy use to a counterfactual, baseline usage scenario in the absence of the program. In the case of a RCT, baseline usage is determined by the control group.

The remainder of this section covers the analysis population, the regression analysis, and converting the savings into an annualized value.

6.1 Participants in the Analysis

The number of participants and controls in the evaluation of savings from behavior modification varies with the program year and fuel type. To be included in the analysis, the customer must have: 9-12 months of pre- and 12 months of post-program usage data, a zip code, and annual usage within designated band.2 Individual records needed to have positive usage, a bill duration > 20 and < 40 days, and usage within one order of magnitude from the wave’s median usage value.

Electric Measure

Program Year Usage Band Number of Participants Number of Controls Number of Cohorts1st Year 5k-7k kWh 95,454 31,779 121st Year 7k-9k kWh 146,082 32,534 121st Year 9k-11k kWh 212,290 84,566 121st Year > 11k kWh 243,368 66,902 122nd Year 5k-7k kWh 70,981 19,665 102nd Year 7k-9k kWh 147,427 32,765 102nd Year 9k-11k kWh 177,605 68,770 102nd Year > 11k kWh 243,559 66,984 103rd Year 7k-9k kWh 44,199 11,691 43rd Year 9k-11k kWh 81,061 40,298 44th Year 7k-9k kWh 40,144 19,968 24th Year 9k-11k kWh 41,077 20,396 25th Year 7k-9k kWh 40,258 20,042 1

Gas Measure

2 One cohort was excluded from the analysis as annual usage fell below 900 therms. Michigan Residential Measures 6 Version 1.5

Program Year Usage Band Number of Participants Number of Controls Number of Cohorts1st Year 900-1200 Therms 82,333 44,214 61st Year >1200 Therms 215,000 66,549 62nd Year 900-1200 Therms 90,216 43,930 62nd Year >1200 Therms 207,160 66,862 63rd Year 900-1200 Therms 47,423 26,845 33rd Year >1200 Therms 34,761 17,309 34th Year 900-1200 Therms 41,110 20,406 24th Year >1200 Therms 34,754 17,309 25th Year 900-1200 Therms 34,761 17,309 1

6.2 Regression Analysis

Navigant used a Linear Fixed Effects Regression (LFER) model to estimate daily per participant savings from behavior modification reports, by usage band and program year, using monthly billing data for DTE Energy’s and Consumers Energy’s HER program participants between 2010 and 2016.3 Formally, the model is:

Where,

ADU¿ is the average daily usage for household i during month tα i is a household-specific fixed effect that captures factors which do not change over

time Post¿ is a binary variable taking value of 0 if month t for household i is in the pre-period, or

1 in the program periodTreatment i is a binary variable identifying if a household is in the treatment (1) or control (0)

groupWaveib is a factor variable identifying whether household i is in wave bCDD¿ is the number of cooling degree days for household i during month tHDD¿ is the number of heating degree days for household i during month tε ¿ is the cluster-robust error term for household i in time t

Coincident peak demand savings for each measure is calculated by multiplying the energy savings rate by a factor of 1.5. This approach is consistent with the approach used historically for determining coincident peak demand savings values for this measure and was developed through observation of Behavior Modification Report energy savings in multiple programs using hourly meter data from AMI.

6.3 Annualizing Savings

To get one annualized savings value, average daily savings is multiplied by 365 days.

7. Estimated Energy Savings over Baseline

3 This methodology is consistent with the methodology the Michigan Energy Waste Reduction Collaborative approved resulting from the Behavior Modification Report Savings Model Calibration Study completed in 2015. Refer to Attachment I.Michigan Residential Measures 7 Version 1.5

ADU❑¿=α i+∑bβ1b (Post¿ ∙ Treatment i ∙Wave ib)+∑

bβ2b (Post¿ ∙Waveib )

+∑bβ3b(CDD ¿∙Waveib)+∑

bβ4b(HDD¿ ∙Waveib)+ε¿

This section provides savings estimates from the regression model described in Section 6.2 and formulas to calculate energy and coincident peak demand savings for the behavior modification report, along with illustrative examples.

Figure 2 illustrates how savings are adjusted to account for savings overlap. The details of the savings overlap are discussed in Section Error: Reference source not found.

Figure 2. Summary of Savings

Source: Navigant

7.1 Regression Savings Estimates

The table below shows the estimates of per customer daily savings from the regression prior to adjusting for double counting. The daily savings divided by the daily baseline is equal to the percentage savings. The standard errors shown in italics indicate the precision of the estimate.

Electric Measures

Program Year Usage Band

Average Daily Savings Per

Participant (kWh)Standard Error

Baseline Daily Usage

(kWh)*

Percentage Savings Per

Participant (%)Standard Error

90% Confidence

Interval (%)**

1st Year > 11k kWh Annual Usage

0.480.03 39.88

1.20%0.08% (1.07%-1.33%)

1st Year 9k-11k kWh Annual Usage

0.320.03 29.46

1.08%0.09% (0.93%-1.23%)


0.210.04 24.44

0.84%0.16% (0.58%-1.10%)


0.120.03 16.69

0.70%0.17% (0.42%-0.98%)

Program Year

Usage Band Average Daily Savings Per

Participant (kWh)

Baseline Daily Usage

(kWh)*


Participant (%)

90% Confidence

Interval (%)**Michigan Residential Measures 8 Version 1.5

Standard Error Standard Error2nd Year > 11k kWh

Annual Usage0.710.04 39.70

1.78%0.11% (1.60%-1.96%)

2nd Year 9k-11k kWh Annual Usage

0.450.04 29.46

1.52%0.12% (1.32%-1.72%)


0.370.06 24.29

1.50%0.23% (1.12%-1.88%)


0.120.04 15.90

0.77%0.24% (0.38%-1.16%)

3rd Year 9k-11k kWh Annual Usage

0.490.05 27.59

1.77%0.19% (1.46%-2.08%)

3rd Year 7k-9k kWh Annual Usage

0.410.10 22.36

1.82%0.43% (1.11%-2.53%)

4th Year 9k-11k kWh Annual Usage

0.360.09 28.94

1.25%0.30% (0.76%-1.74%)


0.560.08 24.76

2.27%0.34% (1.71%-2.83%)


0.500.09 24.68

2.01%0.37% (1.40%-2.62%)

*Baseline daily usage is average daily usage by participants during the post program period plus the estimate of average daily savings.**90% confidence interval is calculated as percentage savings per participant +/- 1.645*standard error, where 1.645 is the z-score associated with the 90% confidence level. Source: Navigant

Gas Measures

Program Year Usage Band

Average Daily Savings Per Participant (Therms)

Standard Error

Baseline Daily

Usage (Therms)*



90% Confidence Interval (%)**

1st Year 900-1200 Therms Annual Usage

0.020.00 3.22

0.36%0.12% (0.16%-0.56%)

1st Year >1200 Therms Annual Usage

0.010.00 3.55

0.45%0.07% (0.33%-0.57%)

2nd Year 900-1200 Therms Annual Usage

0.020.00 3.12

0.56%0.14% (0.33%-0.79%)

2nd Year >1200 Therms Annual Usage

0.020.00 3.52

0.63%0.09% (0.48%-0.78%)

3rd Year 900-1200 Therms Annual Usage

0.030.01 3.14

0.96%0.23% (0.58%-1.34%)

3rd Year >1200 Therms Annual Usage

0.030.01 4.35

0.60%0.19% (0.29%-0.91%)

4th Year 900-1200 Therms Annual Usage

0.030.01 3.06

0.91%0.18% (0.61%-1.21%)

4th Year >1200 Therms Annual Usage

0.030.01 4.21

0.70%0.21% (0.35%-1.05%)

Program Year

Usage Band Average Daily Savings Per Participant (Therms)

Baseline Daily

Usage (Therms)*



90% Confidence Interval (%)**


Standard Error

5th Year 900-1200 Therms Annual Usage

0.030.01 3.89

0.72%0.27% (0.28%-1.16%)

*Baseline daily usage is average daily usage by participants during the post program period plus the estimate of average daily savings.**90% confidence interval is calculated as percentage savings per participant +/- 1.645*standard error, where 1.645 is the z-score associated with the 90% confidence level. Source: Navigant

7.2 Formula to Calculate Savings

For an electric Behavior Modification Report targeting 10,000 customers in the first program year, with average annual usage of 8,000 kWh and 5 coincident peak kW.

Electric Savings are:

672,000 kWh = 0.84% * 8,000 kWh * 10,000

Where,

kWhSavingsRate is the electric energy savings rate according to program year and usage band

CtrlUsage is the 12-month average usage of the control group in the year prior to the current program year

NumHouseholds is the number of consistent households from the HER treatment group4

Coincident Peak Demand Savings are:

DmdSavings(kW )=kWSavingsRate∗CtrlDmd∗NumHouseholds630 kW = 1.26% * 5 kW * 10,000

Where,

kWSavingsRate is the demand savings rate according to program year and usage band CtrlDmd is average demand of the HER control group coincident with system peak

(defined as 3 to 6 pm on the three hottest consecutive non-holiday weekdays in July) in the year prior to the current program year

NumHouseholds is the number of consistent households from the HER treatment group

For a gas Behavior Modification Report targeting 10,000 customers in the first program year and average annual usage of 1,300 Therms.

Gas Savings are:

55,900 Therms = 0.43% * 1,300 Therms * 10,000

Where,

THMSavingsRate is the energy savings rate according to program year4 The number of consistent households from the treatment group is determined by counting all households that are active and receive the full program year of HER treatment. Michigan Residential Measures 10 Version 1.5

CtrlUsage is the 12-month average usage of the HER control group in the year prior to the current program year

NumHouseholds is the number of consistent households from the HER treatment group

For a Dual Fuel Behavior Modification Report targeting 10,000 customers in the first program year, with average annual electric usage of 8,000 kWh and 5 coincident peak kW, and average annual gas usage of 1,300 Therms, the electric, coincident peak demand, and gas savings are the same as above.

8. Potential Savings Overlap

Behavior Modification Reports may increase participation in other utility energy efficiency programs; this additional participation is known as efficiency program uplift and could result in double counted savings if not addressed. To avoid double counting savings, the savings from program uplift are subtracted from behavior modification report savings and attributed to the lifted program.5

The SEE Action Network’s report from May 2012 on best practices for measuring and verifying energy savings from behavioral program identifies the following three steps for measuring savings overlap using a difference-in-difference (DID) statistic:

1. Savings resulting from participation in all other energy efficiency programs are calculated for both the participant and control groups6

2. The difference between participant and control groups is measured

3. Any statistically significant difference in savings is then assigned either to the other program, the program offering behavior modification reports, or split between the programs—in this case all double counted savings were subtracted from the Behavior Modification Report

The general process for removing double counted savings from the savings estimate out of the regression analysis is described in Figure 3. The blue shapes outline the steps for the regression analysis (described in detail in Section 6.2Error: Reference source not found), the green shapes outline the steps for the double counting analysis (described in more detail in the rest of this section). The equation at the bottom of the figure shows that double counted savings are removed from savings coming out of the regression analysis to get the savings value without double counted savings which is recommended for the MEMD.

5 Refer to Chapter 17 of the Uniform Methods Protocol, Residential Behavior Protocol, for additional information uplift and the methods used to address double counting. 6 This does not include programs for which tracking data are unavailable, such as upstream lighting programs. Michigan Residential Measures 11 Version 1.5

Figure 3. Double Counting Adjustment Process

Source: Navigant

The calibration study did not determine program uplift for each cohort. Instead, it combined average program uplift estimates for DTE Energy and Consumers Energy across all study cohorts, weighted by the proportion of participants from each utility.7

The program uplift estimates for DTE Energy were based on a DID statistic. This value is defined as the program’s participation rate change between program and pre-program years for the control group subtracted from the same change for the treatment group.

The program uplift estimates for Consumers Energy were based on a simple difference between treatment and control participation rates during the program year.

To calculate the DID statistic, Navigant subtracted the change in the participation rate in another energy efficiency program between the 12 months after a participant started receiving reports (the post period) and the 12 months before (the pre period) for the control group from the same change for the treatment group. This calculation is shown in Equation 1. The simple difference removes the pre-period participation from the equation.

Equation 1. DID Statistic Calculation( post period treatment group participation−pre period treatment group participation )−(post period control group participation−pre period control group participation )=DID statistic

The DID statistic generates an unbiased estimate of uplift when the baseline average rate of participation is the same for the treatment and control groups, or when they are different due only to differences between the two groups in time-invariant factors, such as the square footage of the residence. The simple difference statistic is unbiased when the baseline average rate participation is the same for the treatment and control groups.

The steps to go from the DID or simple difference statistic to an adjusted per participant savings value are outlined in Equation 2. Multiplying the DID statistic by the number of participants gives the change in the number of people participating in each other energy efficiency program because of the behavior modification report. Multiplying this number of people by the median savings from the other program

7 Estimates were based on evaluations of program tracking data for all programs where data was available. Michigan Residential Measures 12 Version 1.5

gives the double counted savings. The double counted savings is calculated for each other program considered and is then summed across all the programs to get total annual double counted savings. Dividing this total annual number by the total savings generated by the behavior modification report programs gives the double counting adjustment value.

Equation 2. Double Counting Adjustment Steps

∆ ¿ of people participating∈other EE programs=¿DID (SimpleDifference)statistic∗¿of participants∈the smartphonebehavior application

Double counted savings=¿∆ ¿ of people participating∈other EE programs∗mediansavings ¿other EE programs

Double countingadjustment value=Doublecounted savingsTotal savings ¿

Behavior Modification Report program ¿

The double counting adjustment values from past evaluations were combined to produce the following adjustment values:

Electric Gas0.17% 5.04%

The reported savings values were calculated using the following formula:Regression Savings Rate x (1 – Weighted Average of Difference Statistics) = Reported Savings Rate

9. Coincidence Factor

The rate of coincident peak savings is 1 for all measures in this whitepaper.

10. Measure Life

The measure life is 1 year for all measures in this whitepaper.

11. Measure Cost

$6.77 per measure (i.e., cost to deliver the report to one participant for one year)

12. Any Recurring Cost

There are no recurring costs associated with this measure.

13. Relevant Codes and Standards

There are no relevant codes or standards associated with this measure.

14. Ongoing EM&V, Research, and Calibration Planning

Year 6 savings using a complete year of data will be submitted for the 2018 MEMD, pending guidance from the Michigan Public Service Commission.

Year 7 savings will be submitted for the 2019 or 2020 MEMD, pending guidance from the Michigan Public Service Commission.


15. Sources of Information

U.S. Department of Energy, “Chapter 17: Residential Behavior Protocol,” in The Uniform Methods Project: Methods for Determining Energy Efficiency Savings for Specific Measures, 2015. Accessed 22 March 2017, https://energy.gov/sites/prod/files/2015/02/f19/UMPChapter17-residential-behavior.pdf.

SEE Action Network, “Evaluation, Measurement, and Verification (EM&V) of Residential Behavior-Based Energy Efficiency Programs: Issues and Recommendations,” 2012. Accessed 22 March 2017, https://www4.eere.energy.gov/seeaction/system/files/documents/emv_behaviorbased_eeprograms.pdf.

SEE Action Network, “Insights from Smart Meters: The Potential for Peak-Hour Savings from Behavior-Based Programs.” 2014. Accessed 22 March 2017, https://www4.eere.energy.gov/seeaction/system/files/documents/smart_meters.pdf.

16. Attachments

Attachment A: “How to Count Behavioral Savings in Michigan”, presentation given to Evaluation Collaborative on July 17, 2012

o File Name: Attachment_A_HER_Program_Year 1_July_2012_Final.pdf Attachment B: MEMD Year 2 update slides

o File Name: Attachment_B_HER_Program_Year 2_Updated_12_Month_Data_5-29-13_FINAL

Attachment C: MEMD Year 3 update slides o File Name: Attachment_C_HER_Program_Y3_Values_5-29-14_Final.pdf

Attachment D: MEMD Year 4 update slideso File Name: Attachment_D_HER_Program_Y4_Values_final.pdf

Attachment E: MEMD Year 5 update slides.pdfo File Name: Attachment_E_HER_Program_FinalY5_Values.pdf

Attachment F: MEMD Preliminary Year 6 update slideso File Name: Attachment_F_HER_Program_Preliminary_Y6 Values_Preliminary.pdf

Attachment G: “Custom Measure M&V Protocol: PPL Electric’s Opower Energy Education Program,” as approved by GDS Associates under Pennsylvania’s Act 129

o File Name: Attachment_G_CMP_Behavioral_PA-08-2010.pdf Attachment H: Allcott, Hunt. 2011. “Social norms and energy conservation” Journal of

Public Economics.o File Name: Attachment_H_Allcott_Social_Norms_and_Energy_Conservation.pdf

Attachment I: Navigant Consulting, Inc. 2015. “Behavior Modification Report Model Calibration Study”

o File Name: Attachment_I_Behavior_Modification_Report_Model_Calibration_Study.pdf

Attachment J: Steward, James (Cadmus) and Pete Cleff (PPL Electric). “Are You Leaving Peak Demand Savings on the Table? Estimates of Peak-Coincident Demand Savings from PPL Electric’s Residential Behavior-Based Program.” AESP Working Paper, 2014.

o File Name: Attachment_J_Are_You_Leaving_Peak_Demand_Savings_on_the_Table.pdf


https://www4.eere.energy.gov/seeaction/system/files/documents/smart_meters.pdf

https://www4.eere.energy.gov/seeaction/system/files/documents/emv_behaviorbased_eeprograms.pdf

https://www4.eere.energy.gov/seeaction/system/files/documents/emv_behaviorbased_eeprograms.pdf

https://energy.gov/sites/prod/files/2015/02/f19/UMPChapter17-residential-behavior.pdf

https://energy.gov/sites/prod/files/2015/02/f19/UMPChapter17-residential-behavior.pdf

Documents

origin-sl.michigan.govorigin-sl.michigan.gov/documents/.../Appendix...7.docx · Web viewThe Home Energy Report (HER) program, implemented by the investor-owned utilities in Michigan