EEA Annual ReportX(1)S(s5ibn0ff5eln2o55oizkej55))/EEInfo... · Increased Staffing –to increase grassroots marketing efforts, EEA added an EEA Facilitator to reach the underserved

EEA Annual Report

April 1

2013

Arkansas Energy Office

APSC FILED Time: 4/1/2014 2:05:04 PM: Recvd 4/1/2014 1:57:35 PM: Docket 07-083-tf-Doc. 130

1

Table of Contents

1.0 Executive Summary ……………………………………………………………………. 2

2.0 Portfolio Impact ………………………………………………………………………... 7

3.0 Portfolio Programs ……………………………………………………………………... 8

3.1 Residential Education and Information Outreach ………………………………… 9

3.2 Media Promotion …………………………………………………………………. 12

3.3 Commercial and Industrial Education and Information Outreach ……………….. 13

3.4 EEA Program Evaluation ………………………………………………………… 17

4.0 Supplemental Requirements ………………………………………………………….. 19

4.1 Training …………………………………………………………………………… 19

4.2 Lost Contribution to Fixed Cost ………………………………………………….. 21

4.3 Unity Performance Incentives ……………………………………………………. 21

4.4 Challenges & Opportunities……………………………………………………… 21

4.5 Market Maturity ………………………………………………………………….. 21

4.6 Staffing …………………………………………………………………………… 21

4.7 Stakeholder Activities ……………………………………………………………. 21

4.8 Estimation of EE Resource Potential ……………………………………………. 21

4.9 Information Provided to Consumers to Promotion EE ………………………….. 21

Appendix ………………………………………………………………………………………. 22

5.0 Appendix A: Glossary of Terms

6.0 Appendix B: EM&V Contractor Report

7.0 Appendix C: Arkansas Industrial Energy Clearinghouse Report


2

BEFORE THE

ARKANSAS PUBLIC SERVICE COMMISSION

Energy Efficiency Arkansas

2013 Annual Report

1.0 Executive Summary

The Energy Efficiency Arkansas (“EEA”) 2013 report for the EEA Arkansas program is provided

by the Arkansas Economic Development Commission-Energy Office (“AEO”) on behalf of the Arkansas

Oklahoma Gas Corporation, Arkansas Western Gas Company, CenterPoint Energy Resources Corp.,

d/b/a CenterPoint Energy Arkansas Gas, The Empire District Electric Company, Entergy Arkansas, Inc.,

Oklahoma Gas and Electric Company, and Southwestern Electric Power Company (“EEA Utilities” and,

together with the AEO, the “Parties”) for the statewide education program approved in Order No. 7 of

Docket No. 07-083-TF. Although not required by the Arkansas Public Service Commission (“APSC” or

“Commission”) to participate in the First Memorandum of Understanding (“MOU”), the Arkansas

Electric Cooperatives Corporation (“AECC”) agreed to voluntarily contribute their pro-rata share for

expenses contained in the First MOU.

This EEA 2013 Annual Report covers the recommendations filed in the Third Amended MOU

approved by the Commission on December 31, 2012. The report covers the activities for the EEA

Comprehensive program that commenced on January 1, 2013 through December 31, 2013. The EEA

2013 report discusses the EEA Working Group Activities and the four EEA Program Components.

The EEA Comprehensive Program consists of four primary components: Education and

Information Outreach (Residential), Media Promotion, Commercial and Industrial Education and

Information Outreach, and Program Evaluation. The following is an overview of many of the EEA

activities for each of the four components:

I. The EEA’s Education and Information Outreach (Residential) Program was

supported by additional staffing during the second half of 2013. The AEO employed

an EEA Facilitator to focus on the marketing and outreach of the residential program.

The EEA Facilitator focused on marketing the program by presenting to diverse

groups of customers, staffing outreach opportunities, and promoting trade allies

incentives and rebates. By adding the EEA Facilitator, EEA saw an increase in


3

grassroots activities during the last six months of 2013. The EEA outreach staff was

able to distribute a total of 36,110 fact sheets, booklets and CDs through 24

grassroots events. Through EEA’s toll-free phone number, EEA fulfilled requests for

1,901 packets of EEA’s energy efficiency materials. EEA also reprinted 15,000

copies of “Home Energy Project” or HEP booklets. The reprint of HEP booklet was

requested by code enforcement officials, energy professionals and weatherization

staff.

II. The purpose of the EEA media promotion is to make consumers aware of energy

saving opportunities, motivate individuals to reduce energy consumption, provide

information on specific cost-effective energy efficiency measures, and inform the

public how and where to get additional energy information with the ultimate goal of

changing individual or collective behavior. Due to the limited budget resources for

mass media, EEA did not contract any television or radio advertisement for this

program year. Instead, EEA utilized talk show appearances and print advertisements

to reach Arkansas residents. The following publications were used to educate various

communities about EEA: The Stand, News, Energizing Arkansas, Ink Magazine,

Arkansas Living, SEA Life, ARGreen Living Guide, Perspectives, El Latino, Hola

Arkansas, Amigo News, and La Prensa. EEA hosted summer (June 24th-28

th) and

winter (October 28th-November 1

st) “Tighten-Up Week” campaigns to increase

interest in home energy efficiency. In addition, the Director of AEO appeared on

morning radio and television talk shows to emphasize the benefits of energy

efficiency practices and promote utilities’ rebates and incentives.

III. Commercial and Industrial (C&I) Education and Information Outreach is

designed to provide training to school districts, state agencies and large commercial

and industrial sectors. The EEA Commercial and Industrial Education program

trained a total of 220 participants in topics such as Commercial HVAC, Industrial

Compressed Air Systems, Pumping System Optimization, Boiler Operations and

Safety, and Refrigeration Energy Management. The EEA utilities have participated

in commercial training by enrolling and sending utility staff and/or utility contractors.

Many of the utilities have promoted their rebates and incentives during the

commercial and industrial trainings.


4

Another component of the C&I outreach is the Arkansas Industrial Energy

Clearinghouse (“AIEC”); the primary function of the AIEC is to support and promote

energy efficiency within Arkansas’s industrial and manufacturing sector. The AIEC

serves as a technical resource for Arkansas’s diverse industrial and manufacturing

sectors as well. During program year 2013, the AIEC provided technical information

to over 494 customers and utilities and performed 56 site visits which resulted in

energy efficiency recommendations to those manufacturers (see AIEC report

Appendix C).

IV. During March 2013, the Arkansas Energy Office (AEO) contracted with Cadmus to

perform a process evaluation of the 2012 EEA program, and to make

recommendations for improvement. Cadmus worked with AEO and the Independent

Evaluation Monitor (IEM) staff to identify four areas for the evaluation. The four

areas identified were: 1) program design, 2) program performance and

implementation, 3) market effects and 4) data and information tracking. Based on the

evaluation process, the following are some of the recommendations: 1) Improve

communication and coordination between the various stakeholders, 2) C&I increase

technical knowledge and awareness of energy efficiency among participants, 3)

improve program integration with the utilities, and 4) Improve data collection and

tracking to get better information about market effects, track performance, and

conform to evaluation, measurement, and verification (EM&V) Protocols in the

Arkansas Technical Reference Manual (TRM). Cadmus finalized the EEA evaluation

report (Appendix B) on December 31st, 2013. EEA will work to address all findings

and recommendations made by the evaluator.

Highlights

Increased Staffing –to increase grassroots marketing efforts, EEA added an EEA

Facilitator to reach the underserved and hard-to-reach segments of Arkansas’s

population.

The AIEC-EEA added the AIEC to its programs to provide technical assistance to

industrial and commercial customers. The AIEC conducted over 56 site visits to 28

manufacturers during 2013, which resulted in energy efficiency recommendations.


5

Southeast Energy Efficiency Alliance (SEEA) grant award-EEA was instrumental in

obtaining the five hundred thousand dollar ($500,000) grant from SEEA.

EEA Monthly Bulletin-EEA began publishing a monthly bulletin to inform

stakeholders, contractors and participants of upcoming events, meetings, trainings and

other energy related activities.

Program Evaluation-EEA awarded the Program Evaluation contract to the Cadmus

Group. The program evaluation report was completed on December 31, 2013 (see

Appendix B)

Arkansas Industrial Energy Efficiency Finance Forum (AIEEFF)-EEA was a major

contributor to this forum which had over 150 commercial and industrial customers

seeking information on how to finance energy efficiency projects.

What’s working or what needs improvement?

What is Working:

AIEC recommendations of energy and cost-savings opportunities to Arkansas’s industrial

and manufacturing sectors. AIEC site visits to Arkansas industrial and manufacturing

customers increased to 56 which is a 22 visit increase over last year’s levels.

Recommendations made by the AIEC could result in annual cost savings of $1.2 million.

What needs Improvement:

C&I trainings saw a decrease by 124 participants, or 36 percent below 2012 training

levels. The decrease was due to retirements of two of the training coordinators that

facilitated the C & I Trainings. Arkansas Manufacturing Solutions assigned duties from

the training coordinator to their marketing representative. Also the Federal Government

shutdown in 2013 had an adverse effect on C&I trainings because it caused three (3) of

the trainings to be cancelled due to the fact that Arkansas Manufacturing Solutions is

primarily supported by federal government funds. EEA has addressed the low training

attendance concerns with the marketing representative and developed a plan of action to

address the decreased attendance issue.


6

Utilities Participation

The utilities or their contractors have participated in all of the EEA trainings by having account

representatives attend the training and/or by providing rebates and incentives information. The utilities

also provided major support with staff and energy incentives information for trade shows, presentations,

fairs and other events.

Budget ActualElectric Utilities Target Sector Program Type ($) ($)

Entergy Res/C&I Behavior/Education 224,639 408,086 182%

SWEPCO Res/C&I Behavior/Education 34,147 62,033 182%

OG&E Res/C&I Behavior/Education 18,319 33,279 182%

Empire Res/C&I Behavior/Education 1,263 2,294 182%

Regulatory - - -

Total 278,368 505,692 182%

2013% of

RBudget

EE Portfolio Summary by Electric Utility

EE Program Cost Summary

% of Budget Actual % of

Cost Type Total ($) ($) Total

Planning / Design 0% - - 0%

Marketing & Delivery 100% 278,368 505,692 100%

Incentives / Direct Install Costs 0% - - 0%

EM&V 0% - - 0%

Administration 0% - - 0%

Regulatory 0% - - 0%

100% 278,368 505,692 100%

EE Portfolio Summary by Cost Type - Electric2013 Total Cost

Budget ActualNatural Gas Utilities Target Sector Program Type ($) ($)

CenterPoint Res/C&I Behavior/Education 106,304 193,116 182%

SourceGas Res/C&I Behavior/Education 38,773 70,436 182%

AOG Res/C&I Behavior/Education 11,760 21,363 182%

Regulatory - - -

EE Portfolio Summary by Natural Gas Utility2013

% of

RBudget


7

2.0 Portfolio Impact

The purpose of the EEA Program is to cost-effectively deliver relevant, consistent, and fuel

neutral information and training that causes people to consume less energy through energy efficiency and

conservation measures.

Below is the program cost by each utility which includes the budgeted and actual dollars for each

report year.

Electric Utilities:

EE Program Cost Summary

% of Budget Actual % of

Cost Type Total ($) ($) Total

Planning / Design 0% - - 0%

Marketing & Delivery 100% 156,837 284,915 100%

Incentives / Direct Install Costs 0% - - 0%

EM&V 0% - - 0%

Administration 0% - - 0%

Regulatory 0% - - 0%

100% 156,837 284,915 100%

EE Portfolio Summary by Cost Type - Natural Gas2013 Total Cost

Program Budget Actual % Plan Evaluated % Plan Evaluated % Plan Acutal %

Program Year 2011 297,988$ 190,489$ 64% 0 0 - 0 0 - 0 0 -

Program Year 2012 317,952$ 268,137$ 84% 0 0 - 0 0 - 0 0 -

Program Year 2013 224,639$ 408,086$ 182% 0 0 - 0 0 - 0 0 -

Entergy Portfolio Impact 07-083-TFCost Energy Savings (kWh) ParticipantsDemand Savings (kW)


Program Year 2011 45,912$ 29,353$ 64% 0 0 - 0 0 - 0 0 -

Program Year 2012 48,332$ 40,760$ 84% 0 0 - 0 0 - 0 0 -

Program Year 2013 34,147$ 62,033$ 182% 0 0 - 0 0 - 0 0 -

Swepco Portfolio Impact 07-083-TFCost Energy Savings (kWh) ParticipantsDemand Savings (kW)


Program Year 2011 24,211$ 15,458$ 64% 0 0 - 0 0 - 0 0 -

Program Year 2012 25,929$ 21,867$ 84% 0 0 - 0 0 - 0 0 -

Program Year 2013 18,319$ 33,279$ 182% 0 0 - 0 0 - 0 0 -

OG&E Portfolio Impact 07-083-TFCost Energy Savings (kWh) ParticipantsDemand Savings (kW)


8

Gas Utilities:

3.0 Portfolio Programs

The EEA Comprehensive Program consists of four primary program components which are:

Residential Education and Information Outreach

Media Promotion

Commercial and Industrial Education and Information Outreach

EEA Program Evaluation


Program Year 2011 1,696$ 1,086$ 64% 0 0 - 0 0 - 0 0 -

Program Year 2012 1,787$ 1,507$ 84% 0 0 - 0 0 - 0 0 -

Program Year 2013 1,263$ 2,294$ 182% 0 0 - 0 0 - 0 0 -

Empire Portfolio Impact 07-083-TFCost Energy Savings (kWh) ParticipantsDemand Savings (kW)


Program Year 2011 133,181$ 85,106$ 64% 0 0 - 0 0 - 0 0 -

Program Year 2012 150,462$ 126,889$ 84% 0 0 - 0 0 - 0 0 -

Program Year 2013 106,304$ 193,116$ 182% 0 0 - 0 0 - 0 0 -

CenterPoint Portfolio Impact 07-083-TFCost Energy Savings (Therms) ParticipantsDemand Savings (Therms)


Program Year 2011 48,971$ 31,307$ 64% 0 0 - 0 0 - 0 0 -

Program Year 2012 54,879$ 46,281$ 84% 0 0 - 0 0 - 0 0 -

Program Year 2013 38,773$ 70,436$ 182% 0 0 - 0 0 - 0 0 -

SourceGas Portfolio Impact 07-083-TFCost Energy Savings (Therms) ParticipantsDemand Savings (Therms)


Program Year 2011 14,876$ 9,509$ 64% 0 0 - 0 0 - 0 0 -

Program Year 2012 16,645$ 14,037$ 84% 0 0 - 0 0 - 0 0 -

Program Year 2013 11,760$ 21,363$ 182% 0 0 - 0 0 - 0 0 -

AOG Portfolio Impact 07-083-TFCost Energy Savings (Therms) ParticipantsDemand Savings (Therms)


9

The following is a brief summary of the activities and progress of EEA:

3.1 RESIDENTIAL EDUCATION AND INFORMATION OUTREACH

The Residential Education and Information Outreach of the EEA Comprehensive Program is

comprised of the following four activities: (A) updating and reproducing fact sheets; (B) reproducing and

co-branding publications; (C) distribution of information via requests and outreach; and (D) Residential

Energy Codes Training/Energy Star Homes Seminar

a. Updating and reproducing fact sheets

The EEA updated the five fact sheets (Heating, Cooling, Water Heating, Locating and Sealing

Air Leaks, and Lighting and Appliances). Each fact sheet provides customers with an explanation of low-

cost/no-cost actions that they can take within key topic areas and the resulting benefits.

2014 Outlook: EEA will update and reprint 30,000 copies of each fact sheet.

b. Reproducing and Co-branding Publications

The EEA has converted The Home Energy Project (HEP) booklets and the Five Fact Sheets to a

CD for general distribution to customers. EEA reproduced 10,000 CDs for general distribution. The CD

has information regarding utilities incentives such as rebates, energy efficiency products and tips. EEA

also reprinted 10,000 copies of “30 Simple Things You Can Do To Save Energy and Money” booklets.

2013 Activities: EEA updated and reprinted 15,000 copies of the HEP booklet.

2014 Outlook: The HEP booklet and the 30 Simple Things booklet are two of the most requested

energy savings publications that EEA distributes. EEA will update and reprint 15,000 copies of “30

Simple Things You Can Do To Save Energy and Money” for 2014.

c. Responding to Requests for Distribution of Information

EEA hired an EEA facilitator to distribute outreach information to Arkansas consumers. Most

Arkansans obtained EEA material by phone (1-888-524-4567 toll-free) or grassroots events (fairs,

tradeshows, and conferences).

EEA distributed a total of 38,011 items of EEA material. The EEA outreach staff was able to

distribute a total of 36,110 fact sheets and CDs during 24 grassroots events (see events table page 10).


10

EEA’s staff and the utilities were able to participate in the 2013 Arkansas State Fair and several other

events. EEA distributed a total of 1,901 fact sheets, CDs, and other energy saving information through the

toll-free EEA phone number. Also, the utilities distributed energy efficiency incentive and rebate

information at trade shows and other grassroots events.

2013 Activities: EEA participated in 24 events during program year 2013.

2014 Outlook: EEA has added a program facilitator which will enhance the grassroots outreach

and visibility of the program. EEA plans to participate in 35 events with high visibility.


11

d. ENERGY STAR Homes-Builder Seminars

EEA sponsored no Energy Star Homes Seminars in 2013. AEO will contract Centers of

Excellence to administer Energy Star Training for 2014.

2013 EventsEXTERNAL Events EEA information was distributed

Event No. Date Event and Location Sponsor

1 4/1/2013 Assurance Classes-Pine Bluff Pine Bluff CAP Agency

2 4/10/2013 ABA Earth Day -Little Rock Arkansas Building Assoc.

3 4/22-26/2013 DHS Green Week-Little Rock Dept. of Human Services

4 4/23/2013 UAMS Earth Day-Little Rock UAMS

5 4/24/2013 Interfaith Power and Light Event Interfaith Power and Light

6 5/4/2013 Cinco De Mayo-College Station Hola-Arkansas

7 6/12-13/2013 Summer Energy Academy -Fayetteville University of Arkansas

8 6/19-21/2013 Arkansas Municipal League Conference Arkansas Municipal League

9 6/26-29/2013 Arkansas County Judges Meeting-Hot Springs Judges Assoc.

10 6/28/2013 Delta Green Community Forum-Forrest City Delta Citizens Alliance

11 6/30/2013 Arkansas Industrial Energy Efficiency Forum-

Little Rock


12 9/18-19/2013 Arkansas Hospitality Association-Little Rock Arkansas Hospitality Association

13 9/19/2013 Hola Arkansas Expo-Little Rock Hola-Arkansas

14 9/21/2013 Sherwood Fest-Sherwood City of Sherwood

15 9/27/2013 State Employee Benefit Fair-Little Rock State Employees Human Resources

16 10/1/2013 Arkansas School Plant Management Assoc.-Hot

Springs

Arkansas School Plant Management

Assoc

17 10/11-20/2013 Arkansas State Fair-Little Rock Arkansas Livestock Association

18 10/15-17/2013 Sustainable Communities Leadership Summit-

North Little Rock

University of Arkansas

19 10/25-26/2013 Lake Chicot Fall Festival-Lake Village City of Lake Village

20 10/28-30/2013 2013 Arkansas Weatherization Training

Conference- Little Rock

WAP

21 11/5/2013 Interdepartmental Relations Committee-Annual

Conference for Professional Assistants-Little

Rock

Arkansas Professional Assistant

Association

22 11/6/2013 Youth Home, Inc. Health and Wellness-Little

Rock

Youth Home, Inc.

23 11/9/2013 Engineers Make a World-Mountain Homes Donald W. Reynolds Library

24 11/9/2013 Asian Festival Asian Pacific Resource and Cultural

Center

Distribution of EEA Fact Sheets, CD's, HEP's and 30 Simple Things Bookets

1/1/2013-12/31/2013

Name of Material Distributed

Inventory

As of

12/31/2012 Callin 1-800

Grassroots

Outreach

Total

Distributed

by Item

Inventory As

of 12/31/2013

Heating Fact Sheets 16,637 82 1,555 1,637 15,000

Cooling Fact Sheets 16,502 75 1,427 1,502 15,000

Water Heating Fact Sheets 16,502 525 9,977 10,502 6,000

Lighting and Appliances Fact Sheets 16,502 225 4,277 4,502 12,000

Locating and Sealing Air Leaks Fact Sheets 16,502 425 8,077 8,502 8,000

EEA CD's 4,725 81 1,533 1,614 3,111

* Home Energy Project Booklet (HEP) 0 127 2,408 2,535 12,465

30 Simple Thing Booklet 8,477 361 6,856 7,217 1,260

Total 95,847 1,901 36,110 38,011 72,836

* Reprinted Home Project Booklet 15,000 in 2013


12

e. Residential Energy Codes Training and Outreach

EEA sponsored no residential energy codes training in 2013. AEO will revise the training for

program year 2014 to address the updated energy codes.

f. Centers of Excellence

AEO contracted Northwest Arkansas Community College (“NWACC”) and Pulaski Technical

College (“PTC”) as two Building Training Centers to form the Centers of Excellence (“CoE”). The CoE

developed an impressive training infrastructure with ARRA funds. During 2013, EEA contracted with

the CoEs to continue Building Performance Institute Certification Training, Residential Energy Systems

Network Certification Training, Energy Star and Air-Conditioning Contractors of America Training. The

2013 budget amount was $187,460 for a one year contract term.

3.2 MEDIA PROMOTION

The EEA combined radio, television, and printing into one mass media category with a single

consolidated budget. Consolidating media costs into a single budget category allowed the media

contractor to better capitalize on media block buys at a cheaper price and enabled the marketing firms to

allocate media dollars where the benefits to the program were maximized.

The mass media budget was $350,000 which covered print ads, TV and radio spots during the

program term June, 2010 through December, 2013. The media budget for 2013 was $100,000 budgeted.

a. Maintain a Fuel Neutral EEA Website.

The website offers fuel neutral information on energy efficiency measures, practices, resources,

and technologies for all customer classes. The website has direct links to the utility partners’ websites as

well as AEO publications, EEA media advisories, upcoming events, etc.

The EEA website was updated and refreshed to run through December 2013. Over the life of this

campaign, website traffic has been consistent with more than 2,000 unique visitors to the site during paid

media flights. The most visited page after the homepage was the “Tighten Up” Challenge, an interactive

game used to educate consumers on areas of their home where a few changes can mean great cost

savings. Because of the interest and engagement with this interactive game, the average visitor stayed on

the site for more than three minutes - almost double that of most websites.


13

3.3 COMMERCIAL AND INDUSTRIAL EDUCATION AND INFORMATION

OUTREACH

Below is a description of the Commercial and Industrial training opportunities and associated

budget:

Energy Efficiency in Industry Workshops

The AEO, through the EEA, contracted with Arkansas Manufacturing Solutions

to develop and implement an information outreach program comprised of two activities:

1. Energy 101 focuses on topics that are current and relevant and present timely

opportunities for energy and demand savings

2. Energy management Seminars. These seminars focus on energy efficient

technologies and best practices.

2013 Activities: AMS conducted a Refrigeration Energy Management class; two

Establishing Energy and Next Steps for Building Manufacturing Plants; two Lighting and

Energy Efficiency workshops; two Energy Efficiency 101 classes; and two Boiler/Steam

System Efficiency. Overall, a total of 132 participants attended the trainings.

2014 Outlook: EEA will sponsor six Industrial Energy Efficiency workshops for 2014.

Energy Audits for Commercial/Industrial Training

The commercial audit encompasses the total building. The industrial audit

focuses on efficiency improvements in the production process, but also includes a

building envelope assessment. The benefits of the commercial and industrial audit

training are below:

Analyze building envelope-roofs; walls between conditioned spaces;

construction joints and connections; and door and window frames and seals.

Analyze HVAC systems ductwork, fans and blowers, and electrical

connections

Analyze motors and generators

Analyze lighting

Energy behavior


14

The C/I audit gathers energy data and reviews previous utility bills from the

previous year to find areas of the facility that may be using the most energy. Also the

audit reviews the current rate structure, demand and consumption figures, and other fees

for natural gas, electric, and other fuel. Completion of this training prepares energy

auditors, other energy professionals and participants to take the AEE’s certification test

for Certified Energy Auditor (“CEA”).

2013 Activities: No EEA sponsored CEA seminar 2013.

2014 Outlook: EEA will sponsor one CEA seminar during 2014.

Industrial Compressed Air Systems Training

In many industrial facilities, air compressors use more electricity than any other

type of equipment. The impact of inefficiencies in compressed air systems can therefore

be significant. A properly managed compressed air system can save energy, reduce

maintenance, decrease downtime, increase production output, and improve product

quality.

2013 Activities: EEA sponsored three Industrial Compressed Air Systems trainings; a

total of 34 participants attended the trainings.

2014 Outlook: EEA will sponsor one Industrial Compressed Air Systems training for

2014.

Certified Energy Management (“C.E.M.”) – Program for Professional Certification

Certified energy mangers can assist school districts, hospitals, large commercial

facilities, and industries identify unnecessary energy waste. Offering energy based

education to large commercial and industrial facility and plant management personnel

(with the added benefit of and opportunity for professional certification) increases the

likelihood that real savings will be achieved. Additionally, an increased number of

professionals in the energy trades workforce in Arkansas can enhance the state’s potential

for future utility and state incentives for energy improvements that are dependent on

certified professionals.


15

2013 Activities: EEA sponsored two CEM seminars; a total of 27 participants attended

the trainings. Of the 27 participants, 20 students passed the CEM examination.

2014 Outlook: EEA will sponsor two CEM seminars for 2014.

Benchmarking and Performance C/I Training

The US Department of Energy (“DOE”) and the US Environmental Protection

Agency (“EPA”) have developed a nationally recognized and accepted benchmarking

tool: ENERGY STAR® Portfolio Manager (“ESPM”). ESPM is an interactive energy

management tool that allows businesses and industries to track and assess energy and

water consumption across an entire portfolio of buildings in a secure online environment.

2013 Activities: EEA did not sponsor Benchmarking seminar in 2013

2014 Outlook: EEA will sponsor one Benchmarking seminar for 2014.

Pumping System Optimization

The one day workshop covers practical issues involved in field measurements

and electrical data. It offers an introduction to the Pumping System Assessment Tool

(“PSAT”) software which is used to assess the performance of the pump systems.

Participants learn how the software functions, what data is required, how to use the

software when measured data are not available, and what the assessment results mean.

2013 Activities: EEA sponsored one Pumping System Optimization workshop; a total of

11 participants attended the workshop.

2014 Outlook: EEA will sponsor one Pumping System Optimization workshop for

2014.

Motor Systems Management Training

This training helps the end-users gain skills necessary to effectively manage

electric motor systems; the knowledge results in reduced energy costs and increased


16

system reliability. In addition, the workshop provides an overview of DOE’s

MotorMaster+ and MotorMaster+ International software.

2013 Activities: No Motor System Management trainings were conducted in 2013.

2014 Outlook: EEA will sponsor Motor Systems Management training for 2014.

Boiler Operation and Maintenance Training

The AEO contracted with Applied Thermal Engineering Inc. to conduct several

three-day EEA boiler training workshops. The workshops were designed to provide in-

depth training on safe and energy-efficient boiler operation including maintenance,

inspections, and codes and troubleshooting. At the conclusion of the training, the

students were presented with a certificate of completion and credited with 2.4 Continuing

Education Units.

2013 Activities: EEA sponsored two Boiler Operator workshops; a total of 16

participants attended the workshop. Of the 16 participants, 10 passed the Arkansas

licensing exam for Boiler Operators.

2014 Outlook: EEA will sponsor one Boiler Operator workshop for 2014.

Name

2013

Budgeted

Workshops

2013

Completed

Workshops

2013

Attendees

2013

Budget 2013 Cost Balance

School Facility Managers Training & Webinars 6 $67,348 $67,348

Energy Efficiency in Industry Workshops 4 9 132 $63,654 $72,332 -$8,678

Commercial HVACR Load Sizing & Duct Design 2 $24,725 $24,725

Building Commissioning Workshops 1 $20,964 $20,964

Energy Audits for Commercial/Industrial Training 1 $23,000 $23,000 $0

Industrial Compressed Air Systems Training 1 3 34 $20,553 $28,332 -$7,779

C.E.M. Certification 2 2 27 $35,640 $55,640 -$20,000

Benchmarketing and Performance Training 1 $18,500 $18,500

Pumping System Optimization 1 1 11 $18,500 $11,445 $7,055

Motor Systems Management Training 1 $18,500 $18,500

Boiler Operation and Maintenance Training 1 2 16 $12,500 $16,666 -$4,166

Totals 21 17 220 $323,884 $207,415 $116,469

Comprehensive Program C/I Training


17

Arkansas Industrial Energy Clearinghouse

The AIEC is a joint project of the University of Arkansas and AEO. The primary function

of the AIEC is to support and promote energy efficiency developments in Arkansas

manufacturing plants. The AIEC has a full-time engineering staff and part-time students available

to respond to inquiries from industry, at no cost to the participants.

The AIEC has been in existence for two years, has experienced staff, and is well known

and trusted within the state. Energy savings recommendations for companies located across the

entire state have covered a wide spectrum of energy systems, including compressed air, lighting,

process heat, steam/condensation, waste heat recovery, fans, pumps, and more.

AIEC has served 58 different companies in the state of Arkansas with energy efficiency

recommendations and other support. In addition to supporting the manufacturers directly, the

AIEC indirectly supported these companies through interactions with their utility companies and

their energy efficiency programs.

2013 Activities: The Clearinghouse provided technical information to over 494 industrial

customers and utilities and performed 56 site visits which resulted in energy efficiency

recommendations to those manufacturers. Cost of AIEC for 2013 was $168,647.

2014 Outlook: EEA will continue to fund the AIEC for 2014, at a total cost of $168,647.

3.4 EEA Program Evaluation

RFP Evaluators Selection Committee, composed of seven reviewers that represented gas

utilities, electric utilities, Attorney General, APSC, AEO, ACAAA and the IEM, selected the

Cadmus Group to conduct the EEA program evaluation. The Cadmus Group completed the

program evaluation and submitted a report with several recommendations for the EEA program.

EEA will work diligently to address the findings and recommendations which were outlined in

the program evaluation report (Appendix B). Below are some of the evaluator’s

recommendations:

Involve all stakeholders (EEA, vendors, the utilities, plus others as necessary) in the

collaborative development of an annual program implementation plan for EEA.

Develop and implement a consistent and comprehensive brand and messaging strategy

to strengthen long term awareness and recognition, and ultimately support EEA’s goal

to reach and engage key audiences.


18

Solidify partnerships and communications with utility energy-efficiency programs by

formalizing the utilities’ role in developing the training agenda and content as well as

developing the marketing strategy and plan.

Revisit program budgets to strategically prioritize expenditures that generate the

greatest impact for the program.

Develop a database, including participation data and metrics, which allows AEO to

monitor and maintain information necessary for a comprehensive evaluation and to

track performance

2013 Activities: Completed process program evaluation by Cadmus. Program Evaluation cost

for 2013-$84,828.66.

2014 Outlook: EEA will work diligently to address the finding and recommendations which was

outlined in the program evaluation report.

Benefits Cost Results-N/A


19

4.0 Supplemental Requirements

4.1 Training

Training

EXTERNAL TRAINING (contractors, trade allies, consumer groups, etc.)

Event

No. Start Date Class Class Description

Training

Location Sponsor

No. of

Attendees

(A)

Length of

Session

(B)

Training

Session

Man-Hours

(A x B)

Any

Certificates

Awarded?

(Y or N)

# of

Certificates

Awarded

1 March 27, 2013 Lighting & Energy

Efficiency

Workshop

The course is designed

for commercial, industrial

and institutional building

owners, managers, and

facllity engineers. The

training assesses the

lighting usage and ways

to reduce energy cost by

upgrading the building's

lighting.

Searcy EEA 20 7 140 Y 20

2 March 28, 2013 Lighting & Energy

Efficiency

Workshop

The course is designed

for commercial, industrial

and institutional building

owners, managers, and

facllity engineers. The

training assesses the

lighting usage and ways

to reduce energy cost by

upgrading the building's

lighting.

Russellville EEA 13 7 91 Y 13

3 April 18, 2013 Fundamentals of

Compressed Air

Training course designed

to teach facility

engineers, operators and

maintenance staff how to

achieve 15-25% cost

savings through more

effective production and

use of compressed air

Pine Bluff EEA 13 7 91 Y 13


20

Event


Training

Location Sponsor

No. of

Attendees

(A)

Length of

Session

(B)

Training

Session

Man-Hours

(A x B)

Any

Certificates

Awarded?

(Y or N)

4 April 24-25, 2013 Pumping System

Optimization

Training identifies the

importance of energy

conservation and

highlights opportunities'

to optimize pumping

systems for energy

efficiency

Little Rock EEA 11 16 176 Y

5 May 1, 2013 Boiler and Steam

System Efficiency

The training objective is

to teach energy efficiency

techniques with proper

maintain of a steam

boiler.


6 May 2, 2013 Boiler and Steam

System Efficiency


to teach energy efficiency


maintain of a steam

boiler.

Russellville EEA 11 6 66 Y

7 May 6-10, 2013 Certified Energy

Manager Training

The CEM certification

recognizes individuals

who have demonstrated

high levels of experience

competence, proficiency

and ethical fitness in the

energy management

profession

Fort Smith EEA 16 40 640 Y

8 May 22, 2013 Industrial Energy

Efficiency 101

The workshop provided

an introduction to

important energy

management steps of

assessing your current

and past enery

performance-energy data

gathering and tracking,

establishing baselines,

benchmarking, and

analyzing results that

lead to an energy action

plan.

Jonesboro EEA 24 6 144 Y

9 June 5, 2013 Establishing

Energy & Next

Steps for Building

& Manufacturing

Plants

The workshop provide

and introduction to

important energy

management steps of

assessing the current

and past energy

performance.

Fort Smith EEA 15 8 120 Y

10 October 21-23, 2013 Boiler Operation

Safety Training


to teach safety and

energy efficiency


maintain of a steam

boiler.

Jonesboro EEA 10 20 200 Y

11 October 22, 2013 Refrigeration

Energy

Management

The training addresses

the concepts of energy

consumption in industrial

refrigeration systems

Russellville EEA 10 8 80 Y

12 November 5, 2013 Fundamentals of

Compressed Air

Training course designed

to teach facility

engineers, operators and

maintenance staff how to

achieve 15-25% cost

savings through more

effective production and

use of compressed air

Litte Rock EEA 15 7 105 Y

13 November 6-7, 2013 Advanced

Management of

Compressed Air

Systems

The workshop teaches

participates the

advantages of optimizing

their compressed air

systems -higher

productivity, energy

saving, and increase

product quality.



21

4.2 Lost Contribution to Fixed Cost-NA

4.3 Unity Performance Incentives-N/A

4.4 Challenges & Opportunities

4.5 Market Maturity

4.6 Staffing

The EEA has two full time employees-Program Coordinator and Facilitator.

4.7 Stakeholder Activities

4.8 Estimation of EE Resource Potential

4.9 Information Provided to Consumers to Promotion EE

Event


Training

Location Sponsor

No. of

Attendees

(A)

Length of

Session

(B)

Training

Session

Man-Hours

(A x B)

Any

Certificates

Awarded?

(Y or N)

# of

Certificates

Awarded

14 November 20, 2013 Industrial Energy

Efficiency 101

The workshop provided

an introduction to

important energy

management steps of

assessing your current

and past enery

performance-energy data

gathering and tracking,

establishing baselines,

benchmarking, and

analyzing results that

lead to an energy action

plan.

Jonesboro EEA 12 6 72 Y 12

15 November 18-20, 2013 Boiler Operation

Safety Training


to teach safety and

energy efficiency


maintain of a steam

boiler.

Fort Smith EEA 6 20 120 Y 6

16 December 11, 2013 Establishing

Energy & Next

Steps for Building

& Manufacturing

Plants

The workshop provide

and introduction to

important energy

management steps of

assessing the current

and past energy

performance.

Fort Smith EEA 10 8 80 Y 10

17 December 9-13, 2013 Certified Energy

Manager Training

The CEM certification

recognizes individuals

who have demonstrated

high levels of experience

competence, proficiency

and ethical fitness in the

energy management

profession

Little Rock EEA 11 40 440 Y 11

Totals: Events: 17 220 2,757 220


22

Appendixes


23

5.0 Appendix A: Glossary of Terms

Term Definition

ABudget (Approved Budget) This is the budget most recently approved by the Commission.

Annual Energy Savings Energy savings realized in a full year. (8,760 hours)

Benefit Cost Ratio The ratio of the total benefits of the program to the total costs over the life of

the measure discounted as appropriate.

Custom Savings Savings that are derived from custom measures where deemed savings are not

addressed in the currently approved TRM.

Deemed Savings A “book” estimate of gross energy savings (kWh) or gross energy demand

savings (kW) for a single unit of an installed energy efficiency measure that (a)

has been developed from data sources and analytical methods that are widely

considered acceptable for the measure and purpose and (b) is applicable to the

set of measures undergoing evaluation.

Demand The time rate of energy flow. Demand usually refers to electric power measured

in kW but can also refer to natural gas, usually as Btu/hr or therms/day, etc. The

level at which electricity or natural gas is delivered to users at a given point in

time.

Demand Savings Demand that did not occur due to the installation of an EE measure. (non-

coincident peak)

Energy Sales Energy sold by the utility in the calendar year.

Energy Savings Energy use that did not occur due to the installation of an EE measure.

Gross Savings The change in energy consumption and/or demand that results directly from

program-related actions taken by participants in an efficiency program,

regardless of why they participated.

kW A Kilowatt is a measure of electric demand - 1000 watts.


24

kWh The basic unit of electric energy usage over time. One kWh is equal to one kW of

power supplied to a circuit for a period of one hour.

LCFC Energy Savings For the current Program Year, the sum of eligible net energy savings from (1)

measures installed in prior Program Years (8,760 hours) and (2) measures

installed in current Program Year as adjusted for time of installation, weather,

etc. (less than 8,760 hours). Clarification for item (1) above: The savings

reported in the current year should only reflect the current year impact of

measures installed in prior years but, should not include the savings claimed and

reported in prior years.

Lifetime The expected useful life, in years, that an installed measure will be in service and

producing savings.

Lifetime Energy Savings The sum of the energy savings through the measure's useful life.

Measures Specific technology or practice that produces energy and/or demand savings as a

result of a ratepayer’s participation in a Utility/TPA energy efficiency program.

Net Benefits The program benefits minus the program costs discounted at the appropriate

rate.

Net Savings The total change in load (energy or demand) that is attributable to an energy-

efficiency program. This change in load may include, implicitly or explicitly, the

effects of free drivers, free riders, energy-efficiency standards, changes in the

level of energy service, and other causes of changes in energy consumption or

demand.

Net-to-Gross Ratio (NTGR) A factor representing net program savings divided by gross program savings that

is applied to gross program impacts, converting them into net program load

impacts.

Other Savings Savings for which no deemed savings exist and no custom M&V was performed.

Participant Cost Test (PCT) A cost-effectiveness test that measures the economic impact to the participating

customer of adopting an energy efficiency measure.

Participant A consumer that received a service offered through the subject efficiency

program, in a given program year. The term “service” is used in this definition to

suggest that the service can be a wide variety of services, including financial

rebates, technical assistance, product installations, training, energy-efficiency

information or other services, items, or conditions. Each evaluation plan should

define “participant” as it applies to the specific evaluation.

Plan Savings Annual energy savings budgeted by the Utility for the Program Year.


25

Portfolio Either (a) a collection of similar programs addressing the same market (e.g., a

portfolio of residential programs), technology (e.g., motor-efficiency programs),

or mechanisms (e.g., loan programs) or (b) the set of all programs conducted by

one organization, such as a utility (and which could include programs that cover

multiple markets, technologies, etc.).

Program Administrator Cost

(PAC) Test

The Program Administrator Cost Test measures the net costs of a demand-side

management program as a resource option based on the costs incurred by the

program administrator (including incentive costs) and excluding any net costs

incurred by the participant.

Program Year The Year in which programs are administered and delivered, for the purposes of

planning and reporting, a program year shall be considered a calendar year,

January 1 - December 31.

Program A group of projects, with similar characteristics and installed in similar

applications. Examples could include a utility program to install energy-efficient

lighting in commercial buildings, a developer’s program to build a subdivision of

homes that have photovoltaic systems, or a state residential energy-efficiency

code program.

Ratepayer Impact Measure

(RIM) Test

The Ratepayer Impact Measure (RIM) test measures what happens to customer

bills or rates due to changes in utility revenues and operating costs caused by the

program.

RBudget (Revised Budget) This is the Budget the utility used for the Program Year. This budget may be

different from the Approved Budget (Abudget).

Sales as Adjusted for SD

Exemptions

The Utility's 2010 Annual Energy Sales minus the 2010 Annual Energy Sales of

the customers granted self-direct exemptions by Commission Order.

Total Resource Cost (TRC)

Test

The Total Resource Cost Test measures the net costs of a demand-side

management program as a resource option based on the total costs of the

program, including both the participants' and the utility's costs.

TRC Levelized Cost The total costs of the program to the utility and its ratepayers on a per kWh or

per therm basis levelized over the life of the program.


26

6.0 Appendix B: EM&V Contractor Report

ENERGY EFFICIENCY

ARKANSAS

PROGRAM EVALUATION December 30, 2013



27

This page left blank.


28

Prepared by:

Jamie Lalos

Bonnie Watson

Lolly Lim

Contributions from:

Cheryl Winch

Kathleen Higgins

Cadmus


29

This page left blank.


1

Table of Contents Executive Summary ....................................................................................................................................... 3

Introduction .................................................................................................................................................. 5

Program Description ............................................................................................................................... 5

Program Theory ...................................................................................................................................... 6

Process Evaluation Overview ................................................................................................................. 6

Document Organization ......................................................................................................................... 7

Methodology ................................................................................................................................................. 8

Materials Review .................................................................................................................................. 10

Program Stakeholder Interviews .......................................................................................................... 11

Training Participant Interviews ............................................................................................................ 11

Customer Surveys ................................................................................................................................. 12

Key Findings ................................................................................................................................................ 15

Outreach and Promotion ...................................................................................................................... 15

Planning and Design ....................................................................................................................... 15

Program Awareness ....................................................................................................................... 16

Energy-Efficiency Information Sources .......................................................................................... 17

Marketing Materials and Content ................................................................................................. 18

Barriers and Motivators ................................................................................................................. 19

Program Performance and Implementation .................................................................................. 23

Market Effects ................................................................................................................................ 24

Potential Interest ........................................................................................................................... 26

Data and Information Tracking ...................................................................................................... 28

Commercial and Industrial Training ..................................................................................................... 28

Goals and Objectives ...................................................................................................................... 28

Planning and Design ....................................................................................................................... 29

Target Audience and Attendance .................................................................................................. 29

Outreach ........................................................................................................................................ 30

Awareness ...................................................................................................................................... 31

Program Materials and Website .................................................................................................... 31

Training Content ............................................................................................................................ 32

Training Delivery ............................................................................................................................ 34


2

Satisfaction ..................................................................................................................................... 35

Impacts of Training ........................................................................................................................ 35

Barriers ........................................................................................................................................... 38

Conclusions and Recommendations ........................................................................................................... 40

Program Design .............................................................................................................................. 40

Program Performance and Implementation .................................................................................. 41

Market Effects ................................................................................................................................ 41

Data and Information Tracking ...................................................................................................... 41

Areas for Improvement .................................................................................................................. 43


3

Executive Summary

The Arkansas Energy Office (AEO) contracted with Cadmus in March 2013 to perform an evaluation of

Energy Efficiency Arkansas’s (EEA) 2012 program year, and to make meaningful, actionable

recommendations for improvement. This report describes Cadmus’ findings, conclusions and

recommendations, and will inform EEA’s future program operations and planning.

In 2007, the Arkansas Public Service Commission (PSC) called for “utilities to take actions jointly with the

AEO to design, construct, and fund a statewide education program that has a consistent message

promoting the efficient use of electricity and natural gas,” which was named Energy Efficiency Arkansas.

The purpose of this program, according to a number of Memorandum of Understandings (MOU), is to

cost-effectively deliver relevant, consistent, and fuel neutral information and training that causes people

to consume less energy through energy efficiency and conservation measures.

Cadmus worked with the AEO and Independent Evaluation Monitor (IEM) staff to identify four research

areas to for the evaluation: 1) program design, 2) program performance and implementation, 3) market

effects, and 4) data and information tracking. The evaluation team developed research questions

associated with these areas (Table 2). To address these research questions, Cadmus reviewed program

materials; conducted interviews with program staff and training participants; and surveyed customers.

Based on our findings from evaluation activities, the conclusions and recommendations are highlighted

below.

The Commercial and Industrial (C&I) training is effectively increasing technical knowledge and awareness

of energy efficiency among participants, as well as stimulating participants to take actions such as

talking about energy efficiency. The current design of the Outreach and Promotion is meeting the goal of

increasing awareness among residential Arkansas residents. However building stronger awareness of

the program is limited, due to inconsistent branding. Additionally, residential customers lack familiarity

with EEA. Therefore, Cadmus recommends that EEA develop and implement a consistent and

comprehensive brand and messaging strategy.

EEA can improve communication and coordination between the various stakeholders. There is lack of

communication and coordination involved in the EEA program as it relates to objectives, coordination

for promoting trainings, and sharing of program results. To address this, Cadmus recommend that EEA

develop an annual program implementation plan that is collaborative across stakeholders including EEA,

vendors, utilities, and others as necessary.

EEA and the utility energy-efficiency programs could be better integrated. A driving motivator among

attendees’ reasons to sign up for training is to learn about utility programs. For residential customers,

37% of those who implemented energy-efficiency improvements indicated the improvement was done

through a utility energy-efficiency program. Additionally, the most visited section on the website (except

for the homepage) was the incentives page. Therefore, Cadmus recommends that EEA solidify

partnerships with utility energy-efficiency programs by formalizing their role in developing training

agenda and content as well as designing a marketing strategy that meets objectives of all stakeholders.


4

EEA could improve data collection and tracking to get better information about market effects, track

performance, and conform to evaluation, measurement, and verification (EM&V) Protocols in the

Arkansas Technical Reference Manual (TRM). Cadmus recommends EEA develop a database that

includes participant data and metrics, which allows AEO to monitor and maintain information necessary

for a comprehensive evaluation and to track performance. If not cost prohibitive, EEA should also collect

feedback from training participants sooner after the training.

Several other conclusions from the evaluation are:

Despite a limited marketing budget, awareness of EEA or the campaign name, Tighten Up was

approximately 39%;

There are not enough program resources dedicated to outreach for C&I training;

Nearly all stakeholders and training participants are satisfied with the trainings and find them

useful;

The training instructors selected to lead the trainings provided high quality education and were

knowledgeable about the subject matter; and

Not all C&I training attendees thought the level of technical detail was appropriate for them,

and the program materials did not always specify the level of technical detail that would be

covered in the training.


5

Introduction

In March 2013, the Arkansas Energy Office (AEO) contracted with Cadmus to evaluate Energy Efficiency

Arkansas’ (EEA) 2012 program year and to make meaningful, actionable recommendations for

improvement. This report describes Cadmus’ findings and recommendations and will inform EEA’s

future program operations and planning.

In 2007, the Arkansas Public Service Commission (PSC) called for “utilities to take actions jointly with the

AEO to design, construct, and fund a statewide education program that has a consistent message

promoting the efficient use of electricity and natural gas, which they named Energy Efficiency Arkansas.

The purpose of this program, according to a number of Memorandum of Understandings (MOU), is to

cost-effectively deliver relevant, consistent, and fuel neutral information and training that causes people

to consume less energy through energy efficiency and conservation measures.

Program Description The AEO initially launched EEA in November 2007. Since that time, AEO has increased funding and

added components facilitating the program’s evolution into a comprehensive energy-efficiency effort.

Currently, the program is organized into three distinct efforts:

1. Residential Education and Information Outreach: update and print fact sheets, reproduce and

co-brand publications, distribute collateral consumers request through call center and events,

and sponsor ENERGY STAR® home seminars for builders.

2. Media Promotion: use TV, radio, print, and the web to raise awareness and educate consumers

on energy-saving opportunities.

3. Commercial & Industrial Education and Information Outreach (C&I Training): provide

webinars, trainings, and workshops for school districts, state agencies, and the large C&I sectors.

This training may include energy management seminars; school facility and state building

manager training and webinars; and technical trainings, such as Heating, Ventilation, Air

Conditioning, and Refrigeration (HVACR); load sizing; duct design; energy audits; building

commissioning; compressed air systems; energy management certification; pumping system

optimization; building operator certification; motor systems management; and benchmarking.

For the purposes of this evaluation, Cadmus treated the Residential Education and Information

Outreach and Media Promotion efforts as one element (now referred to as Outreach and Promotion)

since they have similar goals, audiences, and planning processes, and are implemented by the same

marketing agency vendor (Stone Ward).

The EEA Program is funded by the seven investor-owned gas and electric utilities in Arkansas:

CenterPoint Energy, Arkansas Oklahoma Gas Corporation, Sourcegas, The Empire District Electric

Company, Entergy Arkansas, Inc., Oklahoma Gas and Electric Company, and Southwestern Electric

Power Company. The program is also voluntarily funded by the Arkansas Electric Cooperatives

Corporation.


6

EEA has two program managers who are responsible for the day-to-day implementation of the program,

management of the program budget, documentation and program status report production, and

attendance at meetings with the Arkansas Public Service Commission, utilities, and cooperative

stakeholders. For the Outreach and Promotion effort, Stone Ward’s responsibilities include

development and execution of the EEA brand and Tighten Up campaign creative, marketing strategies

and plan, events, collateral, website, as well as media planning, negotiating, buying, managing, and

reconciling/reporting.

For the C&I Training effort, Arkansas Manufacturing Solutions (AMS) oversees the day-to-day program

management for the majority of courses (10 out of 12 topics) offered by EEA. The Arkansas

Environmental Federation (AEF) oversees the management and implementation for two course topics.

The responsibilities of AMS and AEF include developing course content, booking facilities to hold

trainings, selecting instructors, and performing outreach to promote the trainings. In 2012, the EEA

program expanded to include the Arkansas Industrial Energy Clearinghouse (AIEC) to help promote

energy efficiency in Arkansas manufacturing plants. According to the 2012 Annual Report, EEA provided

20 C&I training sessions from February to December 2012 with 344 attendees on twelve topics.1

Program Theory The program theory asserts that increasing the availability of and access to information and training on

the benefits of energy efficiency and conservation will lead to changes in individual and/or collective

behavior to reduce energy use.

Process Evaluation Overview Cadmus worked with the AEO and Independent Evaluation Monitor (IEM) staff to develop the following

main research areas, in compliance with Protocol C of the Arkansas TRM.

1. Program Design. Determine the program design effectiveness by assessing the key elements of

goal setting, budget creation, implementation resources, delivery channels, and outreach

strategies.

2. Program Performance and Implementation. Assess the program’s 2012 performance by

examining progress towards goals, customer and contractor awareness of the program and of

energy efficiency, factors influencing participation, and satisfaction among training attendees.

3. Market Effects. Determine the program’s market effects, such as the energy-saving actions

taken by customers who received materials or heard media ads, or internal business changes

made by participants who attended trainings and workshops.

4. Data and Information Tracking. Examine EEA record-keeping practices and assess

thoroughness, identify gaps, and recommend a tracking system that will allow AEO to monitor

and maintain the information necessary for a comprehensive evaluation. 1 Topics include Commercial HVAC, Boiler Operation Safety, Certified Energy Manager, Economics of Energy

Efficiency, Certified Energy Auditor, Fundamentals of Compressed Air, Advanced Management of Compressed Air, Boiler and Steam System Efficiency, Motor Systems Management, Pumping System Optimization, Refrigeration Energy Management, Energy Management Benchmarking; but exclude Residential Energy Code Trainings.


7

Cadmus addressed the research areas through the process evaluation activities listed in Table 1.

Table 1. Process Evaluation Activities

Activity Activity Overview

Materials Review Reviewed marketing and training materials (including outreach and media plan, media

ads, educational collateral, recaps, website, training curriculum, materials to promote

training, etc.)

Stakeholder Interviews Interviewed AEO, marketing agency and training implementer, and select utility staff

about history and marketing plan development process, goals and objectives, target

audiences, communication and coordination, curriculum priorities, perceived customer

response to the effort, internal data management, and areas of specific interest for the

evaluation.

Training Participant

Interviews

Interviewed contractors and end-users who participated in the training about their

satisfaction with the training content and process, how they learned about the training,

future training needs, overall strengths and weakness of the training, and actions that

resulted from the training.

Customer Surveys Surveyed customers about EEA and their energy-efficiency awareness, how they

learned about EEA, recall of specific marketing materials, barriers and motivations,

energy-efficiency actions taken as a result of the effort, factors that influenced

participation, and future interest in energy efficiency and utility programs.

Document Organization The remainder of this report contains the following sections:

Methodology, which contains an explanation of the evaluation tasks and how Cadmus collected

and analyzed data for this project.

Key Findings, which summarizes key results from our process evaluation activities.

Conclusions and Recommendations, which includes Cadmus’ inferences based on the key

findings to determine how the program is performing and identifies opportunities for improving

processes and outcomes.

Appendices, which contain the interview guides, survey instruments, and survey frequencies.


8

Methodology

Cadmus worked with the AEO and IEM staff to identify the four research areas and associated research

questions shown in Table 2. To address these research questions, Cadmus reviewed program materials;

conducted interviews with program staff and training participants; and surveyed customers.

Table 2. Process Evaluation Research Questions and Activities

Research

Objective Research Questions Evaluation Methodology

Program Design How was the program designed?

What are the program goals and objectives?

Materials review

Stakeholder interviews

Who is the target audience and does the program target all appropriate segments?


Materials review

Customer surveys

What are the marketing/advertising/promotion materials and outreach strategies, and how well do they support program objectives?

Materials review


How appropriately do the marketing materials, content, channels, and messaging target intended audiences?

Materials review


Training participant interviews

Customer surveys

How clear, actionable, educative, and motivating are outreach materials and activities?

Materials review



Are marketing and media resources optimized by channel and target audience?

Materials review



Customer surveys

Do the marketing plan, messaging, strategies, and success metrics correlate to the program objectives?

Materials review



Customer surveys

What are the outreach delivery channels and how effective are they at achieving program objectives?

Materials review



Customer surveys

Program

Performance and

Implementation

How did the program progress towards achieving its planned goals?

Materials (data and tracking) review


Is actual spending consistent with planned budgets? Materials (data and tracking) Stakeholder interviews

What is the optimal budget for EEA marketing and training?



How effective are the program implementation processes and operations at achieving program objectives?




9

Research

Objective Research Questions Evaluation Methodology

What barriers exist that prevent the program from achieving goals and objectives?



What opportunities for improvement exist to further enable program staff to achieve program goals and objectives?



Materials review

How did customers hear about the program? Training participant interviews

Customer surveys

What are customers’ preferred marketing and communications channels?


Customer surveys

How are customers motivated to move from awareness to participation?

Customer surveys


Are training participants satisfied with the training? Training participant interviews

Does training participation correlate to contractor satisfaction? If so, how?



Materials review

How does the variety of trainings correlate to participation in programs and customer satisfaction?


Materials review

Are there suggested areas of improvement? Future training needs?


Customer surveys

Market Effects Are participating segments consistent with targeting activities?



What impacts have marketing and promotions had on awareness, knowledge, and satisfaction with EEA?


Customer surveys

What actions have customers/training participants taken as a result of the outreach and training? What factors led to those actions?

Materials (data and tracking)Review Stakeholder interviews


Customer surveys

Data and Information Tracking

Are data captured and reported in an effective manner?

Materials (data and tracking)Review


How effective are the program’s data tracking processes and systems?



Is collected data sufficient to support effective program implementation and evaluation?



Cadmus completed key stakeholder interviews and customer surveys for the evaluation, as shown in

Table 3.


10

Table 3. Summary of Process Evaluation Activities

Activity Description Population* Target Completed

Program Materials

Review

Review all available materials that EEA

designed to support both the Outreach and

Promotion and C&I Training efforts

N/A N/A N/A

Stakeholder

Interviews

In-depth telephone interviews with AEO,

marketing agency and training

implementation staff, and utility staff

N/A 6 8

Training Participant

Interviews

In-depth telephone interviews with C&I

contractors (e.g., HVAC or building

contractors) and end-users (e.g., facility

managers or building operators) who

received program training

197 30 27

C&I end-users 109 15 17

Contractors 88 15 10

Residential Customer

Surveys

Telephone surveys with a random sample of

residential customers across the state 1,492,008 200 203

Total N/A 236 238

* The populations for C&I training participants exclude the residential energy code classes, and the evaluation

team based the counts on the rosters provided by the AEO. These also exclude attendees with missing key

information (such as phone number) or those the team deemed not appropriate to interview as training

participants (such as utility representatives or Arkansas PSC staff).

Materials Review Cadmus reviewed all the available materials that EEA developed to support both the Outreach and

Promotion and C&I Training efforts (listed in Table 4). The evaluation team qualitatively assessed the

materials and website based on the intended purpose, audience, and desired action for each piece.

Cadmus also rated the materials against the following five best practices for energy efficiency

marketing:

1. Clear central message

2. Clear and direct call to action (CTA)

3. Consistent branding

4. Identifiable target audience

5. Visual appeal

Cadmus scored each of these best practices on a four-point scale (1=Not at all; 2=Somewhat; 3=Mostly;

4=With certainty).


11

Table 4. Materials Reviewed

Material Audience

Energy Efficiency Arkansas Website Residential Consumers, C&I Contractors, C&I End-users

Print Residential Consumers

Direct Mail Residential Consumers

Radio Residential Consumers

Television Residential Consumers

Online Ads Residential Consumers

Public Relations Residential Consumers

Fact Sheets, Booklet Residential Consumers

Online Videos Residential Consumers

Training Flyers and Brochures C&I Contractors, C&I End-users

Training Agendas C&I Contractors, C&I End-users

Training Rosters C&I Contractors, C&I End-users

Post-Training Satisfaction Surveys C&I Contractors, C&I End-users

Additionally, Cadmus reviewed planning and recap documents as part of our overall evaluation for

Outreach and Promotion to understand the process of developing the plan, goals and objectives, and

data tracking.

Cadmus’ approach to assessing marketing materials and plans draws on the expertise derived from

implementing, advising, and evaluating award winning marketing initiatives for conservation programs

at many utilities and energy service providers. This experience allows the evaluation team to identify the

most effective marketing and communications strategies and tactics and create a list of best practice

elements against which to assess marketing effectiveness.

Cadmus used a team approach for the review and analysis to ensure it did not base the ratings on

individual perceptions or preferences.

Program Stakeholder Interviews In March and June 2013, Cadmus conducted in-depth interviews with eight program staff members from

various organizations involved in designing, executing, and supporting the effort. This included staff

from the AEO, the program’s marketing agency, training implementers, and utilities. Cadmus originally

anticipated speaking with six stakeholders; however, in order to gain a fuller understanding of the

program’s implementation attributes and its benefits to multiple parties, the evaluation team

determined it was valuable to interview two additional stakeholders.

The team focused the interview topics on program design and delivery, and program partner

communication. Collectively, these interviews provided the team with an opportunity to assess the

program from multiple perspectives and identify areas for improvement.

Training Participant Interviews In March and June 2013, Cadmus conducted in-depth interviews with 27 training participants.

Respondents included 17 C&I end-users, such as building operations managers, facilities managers, and


12

maintenance supervisors. The remaining 10 respondents consisted of contractors who service C&I

facilities such as sales managers and engineers for HVAC contracting or building contracting companies.

Interviews lasted approximately 30 minutes and covered 10 types of trainings offered through EEA

(Table 5). Cadmus aimed to complete 30 interviews across training types and respondent type (15

contractors; 15 end-users). The evaluation team did not achieve as many contractor interviews as hoped

for the following reasons:

A number of people were too busy to provide feedback and asked interviewers not to call back.

A few people had signed up for the training but did not attend.

There were some wrong numbers and companies that had gone out of business.

Several people, with whom interviewers scheduled follow up calls, did not answer their phone

or call back at the appointed time.

Table 5. Completed Training Participant Interviews

Training Population Contractors C&I End-

users Total

1 Advanced Mgmt. of Compressed Air Systems 9 1 1 2

2 Boiler and Steam Systems Efficiency 25 2 2 4

3 Certified Energy Audit 4 1 0 1

4 Certified Energy Manager 18 1 1 2

5 Economics of Energy Efficiency 14 1 1 2

6 Energy Management Benchmarking 20 0 0 0

7 Fundamentals of Compressed Air 15 0 4 4

8 HVAC 8 1 0 1

9 Motor Systems Management 17 0 2 2

10 Pumping System Optimization 9 2 0 2

11 Refrigeration Energy Management 13 1 2 3

12 State Building and K-12 Energy Manager Training 21 0 4 4

Total 173 10 17 27

* The populations for commercial and industrial training participants exclude the residential energy code classes, and the evaluation team based the counts on the rosters provided by the AEO. These also exclude attendees with missing key information (such as phone number) or those the team deemed not appropriate to interview as training participants (such as utility representatives or PSC staff).

Cadmus focused the interview topics on program design and delivery, program awareness among

training participants, feedback on training content and delivery, utility-funded energy-efficiency rebate

programs, and the market impacts of the training.

Customer Surveys Cadmus contracted with Lawrence & Schiller Teleservices to complete 203 telephone surveys with

Arkansas residents using a random digit dial approach.

The evaluation team designed a survey instrument with input from AEO that averaged 12 minutes.

Lawrence & Schiller fielded surveys in June 2013. Cadmus had two main goals for the customer surveys:

(1) to establish a baseline of customer awareness, barriers, motivations, and impressions of marketing

activities; and (2) to identify responses to marketing and education efforts. The survey also included a


13

limited number of supplemental questions regarding household and customer characteristics, which

Cadmus used to inform future segmentation analyses and targeted marketing approaches.

During the survey fielding process, Lawrence & Schiller provided daily extracts of the data, including

respondents’ geographic location and their responses to questions about awareness. Cadmus monitored

the daily extracts to ensure that survey respondents lived in Arkansas, represented areas from across

the state, and that a representative segment was aware of EEA or the Tighten Up campaign.

In the survey response analysis, the evaluation team focused on identifying differences between

respondents who were “aware” of EEA versus those “unaware”, as this is the primary outcome EEA

marketing was to affect. Because several respondents were aware of EEA, Cadmus determined it did not

need quota groups in the survey fielding process.

Figure 1 shows that respondents represented both rural and populated areas across the state, and that

there was a blend of those “aware” and “unaware” in each region. By random selection, Cadmus

achieved a geographically representative sample and had sufficient survey completions to achieve a 95%

confidence interval (±7% precision) for analysis.

Figure 1. Survey Respondent Address Map*

* Red pins = “Aware”; Yellow pins = “Unaware”

Appendix C provides the frequency tables for each survey, outlining the quantity and percent of each

survey response. This report includes charting and descriptive analyses in the Key Findings section for

items that address research questions. Where segments warranted, the evaluation team conducted

cross-tabulation analysis, testing column proportions between groups. Cadmus tested statistically


14

significant differences at the 95% confidence interval (±5% precision) and 90% confidence interval (±10%

precision), highlighting differences only where testing met or exceeded the statistically significant

threshold.


15

Key Findings

This section includes the findings deemed most important or relevant by the Cadmus evaluation team.

The team derived the findings from the materials review, stakeholder interviews, training participant

interviews, and customer surveys performed. This report presents these findings in two main sections:

Outreach and Promotion and Commercial and Industrial Training.

Outreach and Promotion

Planning and Design

Based on feedback from interviews, the evaluation team found wide variations in stakeholders’

understanding of the goals and objectives of the Outreach and Promotion component. When

interviewers asked program stakeholders to identify the objectives of the marketing effort, responses

included increased awareness of the EEA brand, increased awareness of energy efficiency, improved

awareness of utility offerings, customer implementation of energy-efficiency actions, and to help funnel

consumers to energy-efficiency and incentive programs. There was also a mention from one stakeholder

that objectives shifted over time, based on a budget reduction.

According to stakeholders Cadmus spoke with, AEO initially developed the EEA Program plan in 20072.

As the program progressed, AEO drafted new MOUs that were reviewed by the utilities. In previous

years, there were more formal meetings in which details of the program were discussed; however, those

meetings have become less frequent. Currently, AEO staff provides updates during scheduled PWC

meetings or conference calls. Those meetings and calls are typically high level since the meetings involve

many parties from various levels and not all stakeholders are involved in this meeting. The utility

stakeholders suggested that EEA staff set up formal meetings and/or reporting schedules to ensure that

all parties are up-to-date with program activities and can be involved in strategic discussions.

The target audience of EEA’s 2012 residential Outreach and Promotion effort is broad. Stakeholders said

that the goal is to reach all residents within the State of Arkansas. However, according to the outreach

plan developed by Stone Ward, the target audience is Arkansas residents 25 years or older. Cadmus

understands that EEA staff changed the target audience for 2013, based on the need to optimize the

program’s reduced budget.

The types of outreach materials EEA staff produced include TV ads, radio spots, print ads, fact sheets,

do-it-yourself videos about installing energy-efficiency measures, the EEA website, press releases, public

relations activities, online banners, and bill stuffers. The main element of the Outreach and Promotion

effort was a yearlong campaign, featuring the Tighten Up concept to promote awareness and execution

of simple energy-efficient actions and measures. There was also a Tighten Up Week, in which EEA staff

heavily promoted the campaign through radio and TV talk shows featuring the AEO director.

2 The stakeholders were unclear about the original planning process for the EEA Program, as it started in 2007 and current staff members were not part of that planning.


16

The Outreach and Promotion component also includes a residential energy code class implemented by

the Arkansas Home Builders Association. The class was designed to educate builders, code officials,

remodelers, and HVAC and insulation installers about the upcoming changes to the Arkansas energy

code. The Home Builders Association held 11 of these trainings in 2012.

Program Awareness

Recognizing the potential for confusion or lack of specific name recall with Energy Efficiency Arkansas,

Cadmus defined awareness as acknowledgement of having heard of EEA or the Tighten Up slogan.

Approximately four in 10 (39%) Arkansas residents were aware of EEA or Tighten Up. Although less than

20% were aware of both EEA and Tighten Up, the evaluation team classified any respondent as “aware”

who acknowledged having heard of either brand (Figure 2).

Figure 2. Awareness of EEA and Tighten Up

Source: Customer Survey Questions B1 and B2: “Are you familiar with the energy-efficiency

campaign using the slogan Tighten Up?” and “Before this call today, had you heard of Energy

Efficiency Arkansas?” (n=203)

Although classified as “aware,” a majority of these respondents (72%) were not too familiar or not at all

familiar with either EEA or the Tighten Up slogan. Only 7% of respondents had visited the EEA/Tighten

Up website within the past six months. Most of those who visited the EEA website (80%; 12 of 15) found

the information not too useful or not at all useful.

52%

19%

48%

81%

0% 20% 40% 60% 80% 100%

Aware of EEA

Unaware of EEA

Unaware of Tighten Up Aware of Tighten Up


17

Figure 3. Familiarity with EEA or Tighten Up Slogan

Source: Customer Survey Question B3. “How familiar are you with Energy Efficiency Arkansas or

the Tighten Up campaign?” (n=80)

Of those classified as “aware,” television advertising (42%) was the most common source for their

awareness on an unprompted basis, followed by online advertising (24%).

When prompted with the various types of EEA outreach methods, 63% had heard or seen EEA television

commercials (Figure 4). Less than one third of those “aware” had heard of radio commercials (29%),

news stories (26%), EEA website (24%), fact sheets (20%), videos (18%), community events (18%), or

trainings (11%).

Figure 4. Prompted Awareness for EEA Outreach Activities

Source: Customer Survey Question B6. “Are you familiar with any of these outreach methods

from EEA or the Tighten Up campaign?” (n=80; multiple responses accepted)

Energy-Efficiency Information Sources

Regardless of EEA awareness, two-thirds (65%) of all respondents were aware of organizations in

Arkansas that promote energy efficiency. Of those “aware of an energy-efficiency resource,” one-half

63%

29%

26%

24%

20%

18%

18%

11%

0% 10% 20% 30% 40% 50% 60% 70%

TV commercials

Radio commercials

News story

EEA website

Fact sheets

How-to video

Community events

Trainings

26%

46%

21%

6%

0% 10% 20% 30% 40% 50%

Aware

Very familiar

Somewhat familiar

Not too familiar

Not at all familiar


18

(49%) identified utility companies, and one-third (35%) mentioned EEA, the AEO, or a government

organization.

Respondents considered the Internet as a popular resource for seeking out energy-efficiency

information. When asked where they would look for energy-saving opportunities if interested,

respondents (39%) most commonly mentioned they would do an Internet search (using Google, Bing,

Yahoo, or Ask.com). A smaller proportion (18%) indicated they would seek out the EEA or AEO website,

and another 16% mentioned either the EEA or AEO’s toll-free number.

However, consistent with energy use as a low-involvement category, many survey respondents opted

for less proactive methods for receiving information about saving energy. Traditional channels such as

television (38%), radio (16%), and direct mail (16%) were the preferred resources for receiving

information about how to save energy in the home. Online methods were considered secondary

preferences: e-mail (11%), internet (11%), online ads (1%), online groups (1%), and social media (<1%).

Marketing Materials and Content

EEA has a wide range of materials for the Outreach and Promotion component. These include print ads,

TV and radio spots, and informational fact sheets. EEA’s outreach collateral is distributed to customers

who call EEA, as well as through public libraries and grassroots events, such as fairs, tradeshows, and

conferences. EEA tracks the total number of collateral materials distributed through these channels.

Meanwhile, EEA’s marketing agency tracks the reach of radio spots, TV ads, and print ads as well as the

website interaction data.

These materials serve to inform customers about energy efficiency, provide them with tips on how to

implement energy-efficiency measures, and/or direct them to www.EnergyEfficiencyArkansas.org or to

www.tightenuparkansas.com for more information. Table 6 shows the evaluated metrics for each type

of marketing material as well as the average evaluated score for each metric. The full evaluated metrics

chart is in Appendix E.

Table 6. Marketing Materials - Evaluation Metrics and Average Scoring

Material

Clear central

message/

hierarchy of

messages

Clear and

direct CTA

Consistent

branding

Identifiable

target

audience

Visual appeal

Print Ads 1.8 2.0 3.6 3.8 2.8

Direct Mail 4.0 4.0 4.0 4.0 4.0

Radio Ads 3.5 3.5 2.5 3.5 N/A

Digital 3.0 3.3 3.7 3.3 3.0

Press Outreach 3.5 3.7 3.7 4.0 N/A

Informational 3.2 3.2 3.7 4.0 3.0

Misc. Collateral 3.3 3.7 3.0 2.5 3.5

TV Ads 4.0 4.0 1.0 4.0 4.0

DIY Videos 4.0 4.0 1.0 4.0 3.0

Average Rating 3.4 3.5 2.9 3.7 3.3


http://www.energyefficiencyarkansas.org/

http://www.tightenuparkansas.com/

19

Upon review of the materials, Cadmus found that there is inconsistent branding across all Outreach and

Promotion materials. Some of the materials feature Energy Efficiency Arkansas, while others highlight

Tighten Up.

All effective marketing materials have a clear hierarchy of messages, beginning with a compelling

headline message that guides the reader through program/campaign information and benefits, and

narrows to a specific call for action. Many marketers want to include as much information as possible,

however, that can be overwhelming to consumers. EEA materials should follow a clear message

hierarchy and include a distinct call to action that provides customers with clear and intuitive navigation.

For most materials, the target audience is identifiable as residential energy users in Arkansas as well as

visually appealing. Not as visually appealing, program staff members have appropriately aimed the

informational materials at consumers looking for more detailed information. However, program staff

could update the materials to include a consistent look and feel to fit within the overall EEA materials.

Website

It is important for customers to be able to access an easily navigable website that provides clear

information and steps for participation or action. Most of EEA’s marketing materials direct the reader to

the homepage, www.EnergyEfficiencyArkansas.org, to learn more about lowering energy costs or

applying for a rebate. While the website features a wide range of information and resources, it mostly

consists of a passive repository of information for users to view and absorb. It also includes a large

amount of links back to its own pages and/or materials, which can feel repetitive or overwhelming. The

Tighten Up Challenge and Incentives pages allow the user to take action, however, it is largely

unintuitive to navigate and specifically know where to start. EEA should consider developing a

messaging flow that is in line with how customers search for information, use drop down menus to

allow for easier search, and revisit the website navigational structure and Google Analytics flows to

ensure a streamlined and easy-to-use customer online experience.

From Jan 1, 2012 – Dec 31, 2012, there were approximately 12,800 views to the website. The

homepage, www.EnergyEfficiencyArkansas.org, was the most visited page (30%), followed by the

Incentives page (10%). Also, Google Analytics showed that the Incentives page is the second most visited

page on the website with a 79% bounce rate, which is higher than the overall website average bounce

rate of 56%. The bounce rate of a webpage denotes the percent of first-time visits to that page, in which

the user did not interact with any other component of the website and left the site. The higher than

average bounce rate for the incentives page suggests that users enter the EEA site through this page

then leave immediately—possibly to access utility sites, which are linked to the webpage.3

Barriers and Motivators

Nearly half (45%) of all respondents said saving energy was very important when considering monthly

household expenses.

Considering that some respondents may have already made energy-efficient improvements to their

home, the evaluation team asked them to rate how efficient their home is currently. A majority of

3 The website was not setup to track clicks to the utility websites so Cadmus is unable to track this metric.




20

respondents said their home was somewhat efficient, having some energy-efficient features, but it could

be more efficient. One quarter (26%) of respondents said their home was already very energy efficient

in that it was newly constructed or recently renovated with energy-efficient equipment. This 26%

represents a marketing challenge, in that they see no need for further energy-efficiency home

improvements.

The evaluation team also found this barrier in respondents’ challenges to saving energy. Ten percent

said their home was already somewhat efficient and 20% said they had no challenges in saving energy.

Another cohort (17%) indicated challenges in getting cooperation from other household members.

Twelve percent said they found it difficult to change personal habits associated with energy use.

Significantly more “aware” respondents (14%) said their home was very inefficient compared to those

“unaware” of EEA (5%). This may point to a relationship between EEA awareness and awareness of the

factors that contribute to inefficiency as well as improvement opportunities.

Figure 5. Current Home Energy-Efficiency Level

Source: C4. How energy-efficient would you say your home is currently? (n=200)

Attitudes toward new technologies represent another potential barrier to energy-efficiency adoption. As

many of the equipment rebates and improvements are available through new technologies,

receptiveness to energy-efficiency may correlate with Early Adopter attitudes. One theory4 about how

new products become adopted in a marketplace uses a model to illustrate various levels of new product

adoption. This model postulates there are five segments: Innovators, Early Adopters, Early Majority, Late

Majority, and the last to adopt new technologies, Laggards. These five segments are present in a general

4 Everett M. Rogers, Diffusion of Innovations, Fifth Edition 2003, Free Press, New York.

8%

26%

40%

26%

0%

10%

20%

30%

40%

50%

C4: How energy-efficient would you say your home is currently?

Very inefficient (older home with less efficient heating/cooling and appliances)

Somewhat inefficient (has few if any efficient features and has room for improvement)

Somewhat efficient (has some energy efficient features but could be more efficient)

Very efficient (new home or recently renovated with energy-efficient equipment)


21

population for any new product in proportions that resemble a normal distribution, with the majority

(68%) tending to exhibit attitudes typical of the Early and Late Majority.

Figure 6. Rogers’ Diffusion of Innovation Model

Survey respondents in Arkansas, however, tended to endorse statements associated with later stages on

the diffusion of innovation curve5, with the largest segment, one third (33%), expressing attitudes most

closely associated with Laggards. Roughly 15% could be considered in the earlier stages of innovators

(8%) or early adopters (7%). While diffusion of innovation theory provides context for the uptake of new

technologies over time, it does not address other key factors of user acceptance. Although self-report

may be one limiting factor in classifying respondents according to the Rogers model, other barriers to

diffusion of energy efficiency technologies are also likely at work in this Arkansas population.

Figure 7. Diffusion of Innovation Attitudes

Source: C2. Which of the following statements comes closest to how you feel about new

technologies? (n=203)

5 Diffusion of Innovations takes a divergent approach from other theories of behavior change. The focus is on

the evolution of products as they better meet the needs of individuals and groups, instead of focusing on changing individuals’ behavior. Rogers postulates that change happens not with people, but with the innovations themselves and their role in the marketplace. Everett M. Rogers, Diffusion of Innovations, Fifth

Edition 2003, Free Press, New York.

33%

20%

27%

7%

8%

0% 5% 10% 15% 20% 25% 30% 35%

Laggard: I prefer tried and true methods overnew technologies

Late Majority: I get on board with newtechnologies to keep from falling behind

Early Majority: I like to wait until the bugs areworked out before I invest in new technologies

Early Adopter: I usually invest in newtechnologies before most of my family and…

Innovator: I pay a premium to be the first to tryout a new technology

C2: Which of the following statements comes closest to how you feel about new technologies such as cell phones or electronic

gadgets?


22

This distribution was similar for “aware” and “unaware” respondents, with no statistically significant

differences between groups.

Although respondent attitudes generally appear less than receptive to new technologies, understanding

this barrier may provide useful approaches and tone for developing messages that can speak to this

mind set.

Additionally, respondents cited several other barriers that made saving energy in their homes

challenging. The evaluation team presented this question as an open-ended item and coded similar

responses. The largest single category (20%) was that they had no challenges to saving energy. The

remaining 80%, however, indicated other members in the household, difficulty changing personal

habits, not knowing what to do, an already efficient home, and the expense of home improvements

contribute to challenges they face in saving energy.

Figure 8. Challenges with Saving Energy

Source: C5. What challenges if any do you face in saving energy in your home? (n=191)

Cadmus asked respondents to rate their agreement/disagreement with several attitude statements.

Although responses generally support interest in and willingness to consider saving energy, a barrier

around home comfort priorities was also evident (Figure 9).

Don't know what to do/lack of

information, 10% Can't control use by other members of

the household, 17%

Hard to change personal habits, 12%

Can't afford it/too expensive, 10%

Too hard to install/implement,

5% Not confident it will

save energy/be worth it, 1%

Not a priority, 5%

Home is already pretty efficient, 10%

Challenges with older, inefficient

home, 7%

Renter - limited ability to make

changes, 1%

Disruption to home/mess involved

with installing improvements, 1%

No Challenges/None, 20%

OTHER, 15%


23

Figure 9. Respondent Attitudes (All)

Source: C1 A-I (n=203)

Cadmus also observed several differences between “aware” and “unaware” respondent attitudes. A

significantly greater proportion of “unaware” respondents strongly agreed with the following challenges

to savings energy:

I would like to save more energy but do not know where to start (22% of ”unaware” strongly

agree/9% of “aware”)

I have already done as much as possible to save energy in my home (36% of “unaware”/22% of

“aware”)

Energy-efficiency products are too expensive (27% “unaware”/9% “aware”)

I’ve tried a few things to save energy but have not seen any real savings on my bill (17%

“unaware” versus 8% “aware”)

These differences were statistically significant at the 95/5 confidence level.

Program Performance and Implementation

The 2012 budget for the EEA Program overall was $755,201. The program spent $636,880, or 84%, of

this total allotted budget in 2012. According to the program expense documentation provided for this

14%

17%

20%

31%

33%

34%

51%

55%

58%

39%

41%

46%

46%

35%

35%

38%

32%

37%

38%

25%

26%

19%

27%

22%

9%

12%

4%

0% 20% 40% 60% 80% 100%

C1H: I've tried a few things to save energy, but havenot seen any real savings on my utility bills.

C1D: I would like to save more energy but do notknow where to start.

C1G: Energy-efficient products are too expensive

C1F: I have already done as much as possible to saveenergy in my home

C1E: I always shop for the lowest prices, even if ittakes more time

C1I: I actively look for ways to reduce my carbonfootprint

C1B: Using energy to keep the home comfortable ismy top priority

C1C: Saving energy helps the environment

C1A: It is important to conserve energy as much aspossible.

Strongly Agree Somewhat Agree Somewhat Disagree Strongly Disagree


24

evaluation, program staff used 27% of the total expense for outreach, marketing, and the EEA website,

and 43% of the total expense for the trainings offered by EEA6.

There is no formal procedure for collaboration between the EEA Program and other Utility programs.

There is some collaboration with the utilities in developing materials and performing outreach, which

includes co-branding bill stuffers, placing utility links on the EEA website, and jointly staffing outreach

events within individual utilities’ services territories. The utilities also distribute some of EEA’s materials,

such as fact sheets, to their customers. However, there is less coordination and collaboration from a

strategic and planning perspective.

In prior program years, there were quarterly meetings in which program managers reported on program

status, accomplishments, successes, and lessons learned throughout program implementation.

However, in 2012, formal communications between the EEA program management and the utilities has

decreased. According to stakeholders, the recent meetings have generally served as an opportunity for

the EEA program managers to inform the utilities on the status of the program. Based on interview

feedback, there is a desire to increase collaboration in planning to discuss possible efficiencies and

benefits that could cross stakeholder efforts. For example, it was determined that an objective of EEA

would be to help funnel consumers into utility incentive programs, then this would be an opportunity to

possibly coordinate some tactics. Another example cited was in reference to events that EEA hosts and

staffs. EEA staff has a desire to work with the Utilities to help staff the events, however, many times EEA

did not plan those events within the territories of the Utilities that Cadmus interviewed.

Market Effects

Overall 37% of survey respondents indicated they had made energy-efficient improvements to their

home within the past year. Significantly more of those aware of EEA (46%) made improvements

compared to those unaware (31%).

Of those that had made improvements, the most common were installing compact fluorescent lamps

(CFL) or LED light bulbs (27%), purchasing an efficient window air conditioner (27%), and installing an

energy-efficient central air conditioner (26%).

6 The documentation received did not note whether this expense included costs for the Energy Code training,

which is a component of the Residential Outreach and Promotion effort, and not a part of the C+I trainings.


25

Figure 10. Energy Efficient Investments Made in the Past Year

Source: D2. What did you purchase or install? (n=73)

Nearly one quarter (22%) indicated they had read or heard about the improvement they made through

EEA. About the same proportion (21%) said the EEA information was somewhat to very important in

their decision about which improvement(s) to make.

A little over one-third (37%) of those who had made improvements indicated they had received a rebate

or incentive from their utility.

Significantly more aware respondents indicated EEA as an important factor in their improvement

decision:

Significantly more of the “aware” respondents (31%) said they had heard about the

improvement through EEA (versus “unaware” 14%)

Significantly more “aware” respondents (12%) indicated information from EEA was very

important in their decision (versus 0% of “unaware”)

Significantly more “aware” respondents (47%) received an incentive/rebate from their utility for

the improvement made (versus 26% of “unaware”)

Significantly more “aware” respondents (57%) had purchased a CFL within the past year (versus

41% “unaware”)

3% 3%

4% 4% 4% 4%

5% 8%

10% 11%

15% 16%

18% 18%

26% 27% 27%

0% 5% 10% 15% 20% 25% 30%

Purchased dishwasher

Had ducts sealed

Purchased an efficient light fixture

Installed window film/plastic on windows

Installed air infiltration/ air sealing measures

Had a home energy audit

Installed efficient faucet aerator or low-flow…

Installed an energy efficient water heater

Tuned up their air conditioning or heat pump

Purchased energy efficient refrigerator

Installed energy efficient windows

Other

Purchased efficient clothes washer and or dryer

Added insulation

Installed an energy efficient air conditioning

Purchased/installed CFLs or LED light bulbs

Purchased an energy efficient window AC


26

As some of the energy saving tips provided by EEA address behaviors, the evaluation team also included

questions to assess respondents’ engagement level in typical energy-saving behaviors. Figure 11 shows

the type of behavior and level of engagement endorsed by all respondents.

Figure 11. Energy Saving Behaviors

Source: D10 A-G (n=189-202)

Cadmus found no statistically significant differences on energy-saving behavioral actions between

“aware” and “unaware” respondents; therefore the impact on behaviors has yet to be evident in the

general population.

Potential Interest

One-third (33%) of all respondents were somewhat to very interested in receiving information about

EEA or the Tighten Up campaign in the future. Interest levels were similar between “aware” and

“unaware” respondents, with slightly more “unaware” than “aware” respondents being not at all

interested in receiving more information.

Most respondents (69%) said they did not have plans to make energy-efficient upgrades to their home

in the next 12 months. Thirteen percent said they did have plans and another 4% were considering

upgrades but did not have specific plans.

24%

30%

32%

35%

36%

37%

70%

48%

44%

56%

39%

30%

40%

28%

28%

25%

12%

26%

34%

22%

1%

0% 20% 40% 60% 80% 100%

D10F: Turn down water heater temperature

D10E: Take shorter showers

D10B: Wash laundry in cold water

D10G: Use advanced power strips

D10C: Installed/use a programmable thermostat

D10D: Change thermostat settings to lower heatingtemp when sleeping or not at home

D10A: Turn off lights in unoccupied rooms

Always Sometimes Never


27

Figure 12. Energy-Efficient Improvement Plans in Next 12 Months

Source: D9. Do you have plans to make any energy-efficiency upgrades to your home in the next

12 months? (n=203)

Cadmus analyzed differences between those who had definite plans to make additional energy-

efficiency upgrades to their homes compared to those who did not have plans. The evaluation team

considered the characteristics of those with plans a target profile. Those with plans tended to live in

larger homes (2,500 to 3,000 square feet); have lived in their home for a shorter period of time (less

than 20 years); and have Oklahoma Gas as their natural gas utility. Though most live in single family

homes, more condominium dwellers were among those that had plans for upgrades than among those

without plans.

Although the group with plans have made past improvements in similar proportions to those without

future plans (44% versus 31%), they were more likely to have heard about the improvements they

already made from EEA, and more likely to have received a rebate or incentive for the improvement(s)

they made previously.

Further, they are more likely to recognize energy-efficiency opportunities for their home, to consider

quality over price, and would likely seek out additional information by calling the EEA toll-free number.

Another potential target group includes those that have considered making energy-efficiency upgrades

but do not have specific plans or timing for those upgrades. This group was more likely than other

respondents (those with or without any plans) to have already made an improvement within the past six

months. They also endorsed environmental attitudes and were more likely to have made energy use

behavior changes. While this group is likely “already committed,” members of the group could consider

deeper levels of energy-efficiency upgrades.

Yes, 13%

No, 69%

Considering but have no set plans, 4%

Not sure/Don't Know, 8%

Other, 5%


28


It is a best practice in marketing to identify and track key performance indicators to measure the

performance of marketing campaigns in engaging audiences and inducing the intended audience

response. The EEA campaign tracks traditional marketing and media performance metrics, including

media impressions (including reach and frequency), click throughs, cost per click, amount of collateral

distributed, and events attended. The campaign also tracks basic website analytics including homepage

visits, unique visitors, average time spent on site, and referring sites.

Many efficiency marketing efforts are also starting to use evaluative metrics, which can get closer to

calculating and identifying the sales leads generated by a specific ad. When coupled with traditional

media metrics, this allows the full marketing performance story to be told and for the program manager

to plan and execute the most cost-effective, impactful marketing program. These additional tracking

metrics can be added to marketing materials and advertising, such as unique promotional codes or

URLS, a specific call to action with a unique or mask URL, a special toll-free number to call, etc. can help

marketing managers better track audience response to discrete tactics, ultimately allowing marketing

resources to be channeled to ensure the highest return on investment. In addition to these tracking

metrics, audiences can be asked “how did you hear about us?” type questions through social media

polls, by call center operators, or mini-surveys on the EEA website, and responses tracked in a program

database.

In speaking with a utility program manager, there is effort currently in place to track participant data

between EEA staff and the utilities. While the main objectives of the EEA Program were initially to

increase awareness of energy efficiency overall, according to the MOU dated December 4, 2012, there

may be refinement of activities to better support other utility programs. If this results in a more focused

objective of promoting utility programs, it would require increased coordination of outreach efforts to

differentiate how customers heard about EEA and/or the relevant utility program(s) to track

effectiveness of the various campaigns.

Currently, the only reports developed for this program include a marketing campaign recap developed

and provided by Stone Ward, and the annual report that the AEO submits to the Arkansas PSC. There

are also verbal updates at Parties Working Collaboratively (PWC) meetings. As part of the Arkansas TRM,

EM&V Protocol C3 includes ‘Documentation of program tracking methods and reporting formats’ as an

area of investigation in a process evaluation. There is no formal database that tracks this data, therefore

the Conclusions and Recommendations section includes a set of metrics recommended for tracking and

reporting.

Commercial and Industrial Training

Goals and Objectives

There seems to be an inconsistent understanding among stakeholders of what the goals and objectives

are for the C&I training. The MOU dated November 3, 2010, states that the objective of the C&I

Trainings is to increase awareness of energy-efficiency practices and technologies among school

districts, state agencies, and the large commercial and industrial sectors.


29

However, when asked about the goals and objectives of the C&I training, stakeholders reported a much

wider variety of responses:

Increase sharing of training concepts within companies

Incorporate training concepts into training participants’ business practices

Disseminate information about energy-efficiency to manufacturing and commercial facility staff

to help reduce energy use and save money

Increase awareness and participation in utility-funded energy-efficiency incentives

Build a trained and certified work force to drive participation in utility-funded programs

Promote implementation of more energy-efficient projects

Promote follow-on participation in other training sessions

Reach approximately 20 attendees per training session

In addition to unclear and undocumented goals, the program is lacking measurement to track the

performance of goals with the exception of session attendance. Similar to the Outreach and Promotion

component, the Conclusions and Recommendations section includes a set of metrics recommended for

tracking and reporting.

Planning and Design7

Stakeholders reported that prior to the development of the Second Amended MOU, the Arkansas PSC

requested that EEA develop more comprehensive and extensive trainings within the program. To

accomplish this, Energy Office staff said they added more industry-specific training topics, chosen

through collaboration with the utilities, and by leveraging feedback from various utility trade ally

networks to determine topics that would be most useful and relevant.

Feedback from staff was that stakeholders can discuss planned training sessions and the utilities can

provide feedback on these updates at ongoing PWC sessions. EEA staff also invites the utilities to speak

directly and informally with AMS staff, who manage the trainings, to provide suggestions on future

topics.

Target Audience and Attendance

According to the MOU dated November 3, 2010, the commercial and industrial training sessions target

school districts, state agencies, and large commercial and industrial sectors. Based on interviews with

stakeholders, there does not appear to be a common understanding among stakeholders of who the

specific target audiences are within the commercial and industrial sectors. For example, several

stakeholders said the program intended to reach contractors, but others thought it aimed to reach end-

users. Based on feedback from AEO, both audiences are targets for the training, however this direction

was not clear among stakeholders. Outreach efforts mirror this as some stakeholders targeted outreach

efforts at only end-users, while others aimed outreach targeted both contractors and end-users.

7 The general planning and design of the C&I Training program was developed in conjunction with the Outreach

and Promotion program, and is discussed in detail in the Planning and Design section on p. 15.


30

Cadmus performed an analysis of the training rosters provided by EEA. The evaluation team classified

training participants according to several categories including:

End-users, which include representatives from any commercial and industrial facilities who are

not contractors, such as building operators, maintenance managers, engineering managers at

manufacturing facilities, schools, hospitals, government agencies, etc. (49%)

Contractors, which includes commercial and industrial HVAC contractors, boiler operators,

suppliers, builders, etc. (26%)

Utilities, including utility energy-efficiency program implementation staff, utility program staff,

and municipalities. (22%)

Regulators, including staff from the Arkansas Public Utilities Commission. (2%)

Unidentifiable attendees because they did not list their business name, or they did not provide

enough information otherwise to categorize them into any specific group. (5%)

Commercial and industrial end-users represent about half of the population, and contractors represent

just over a quarter of participants. Utility staff accounted for a large portion (nearly a quarter) of

attendees.

Figure 13 shows the different types of training attendees based on our analysis of the participant rosters

Cadmus received from the EEA.

Figure 13. Percent of Training Attendees by Type

*Some training attendees could not be identified because they did not list their business name,

or they did not provide enough information otherwise to categorize them into any specific group.

Outreach

EEA provides outreach for the commercial and industrial training sessions through distributing flyers and

brochures via e-mail or regular mail. Additionally, EEA recruits training participants through

interpersonal networking and, for some courses, reaching out to contractors using lists derived from the

Arkansas Department of Labor and the Arkansas Department of Health.

26%

49%

22%

2% 5%

Contractor

End-User

Utility

Other

Unidentifiable*


31

EEA does not proactively provide information to the utilities or other stakeholders about upcoming

training opportunities, who signed up (pre-training), or who attended (post-training). The utilities

Cadmus interviewed notify their end-users and/or trade allies about the trainings through e-mail or

direct outreach, but they do not always know about upcoming trainings and AMS does not reach out to

them unless there is very low anticipated participation. Furthermore, the utilities do not use consistent

e-mail distribution lists for outreach. One utility reported sending the e-mails for some training sessions

to commercial and industrial end-users only, while another utility said they sent the training information

to both end-users and their network of commercial and industrial trade allies.

Stakeholders cited the largest barrier to program success is constrained resources for outreach and

marketing. The allocated program resources are not sufficient to allow AMS staff to manage the day to

day activities and spearhead outreach. As a result, participation in the trainings is typically lower than

goal, outreach strategies are not consistently coordinated across stakeholders (e.g., between the

utilities and the EEA), and details of each training (e.g., scheduled training dates, attendee lists, follow-

up, etc.) are not shared in a consistent way among stakeholders.

Awareness

Most respondents first learned about the training through an e-mail (56%) or a professional organization

(19%). Of those who received an e-mail, most (25%) recalled receiving the e-mail from AMS. Other

professional organizations that disseminated information about the training included Arkansas Industrial

Machinery, CLEAResult, the Arkansas Environmental Federation, and Mid-South Steam Boiler and

Engineering Company.

Respondents said they signed up for the training to expand their technical knowledge (69%), to improve

their qualifications (19%) or to learn about energy-efficiency programs (19%).

When asked how they would like to hear about training opportunities in the future, nearly all (93%) of

respondents said they would like to learn about trainings through e-mail. Two respondents would like to

learn about training opportunities through the EEA Website. One respondent mentioned that they did

not think the calendar on the EEA website was very functional because it was not current. The

respondent thought having a central website that was current would be helpful.

Program Materials and Website

The flyers and brochure effectively communicate the training details and target appropriate audiences.

In terms of visual consistency, one training brochure produced by the AEF is very different from the

flyers produced by AMS, so some people may not equate all trainings as having come from EEA. The EEA

branding on the flyers/brochure consists of a small EEA logo. The flyers lacked a consistent “look and

feel” compared to residential EEA outreach materials such as print-ads and bill inserts. The flyers and

brochure all include a call to action to register for trainings via phone or online.

Some materials feature a stronger call to action than others, by noting seating limitations and

encouraging readers to register right away, while most materials state “Register” followed by contact

information. The EEA website does not include much information about the commercial and industrial


32

sector training sessions. The two relevant pages on the website include the Events and Training page,

and the Business section.

The Events and Training page features a link to an events calendar, but this calendar is not user-friendly.

Upon first glance, it looks as though the calendar holds no information. However, users must search for

a specific topic in a text box and/or in a drop-down menu in order to see a list of events or training

sessions on that topic. This process is not user-intuitive and may lead to missed opportunities in

promoting training events.

The Business section does not include information that is relevant to the commercial sector. It includes

links the website’s Resource page, which features resources aimed at residential customers. It also

features links to the three different utilities’ residential weatherization and/or energy audit program

pages.

Table 7 shows the evaluated metrics of each of the program materials as well as the overall evaluated

score for each metric.

Table 7. Program Materials – Evaluation Metrics and Scoring

Program Materials and Website

Evaluation Metrics and Scoring*

Clear and

comprehensive

details

Clear and

direct call

to action

Identifiable

target

audience

Identifiable

EEA

branding

Visual

Appeal

EEA Website - 2013 3 2 4 3 3

CEM + CEA brochure 4 4 4 2 2

Benchmarking flyer 3 3 3 2 4

Motors flyer 4 3 3 2 4

Pumping flyer 4 4 3 2 4

Spring Energy Seminars flyer 4 3 3 2 4

State and K-12 flyer 4 2 4 2 3

Refrigeration flyer 4 3 4 2 4

Economics of Energy Management

flyer 4 3 2 2 4

Fundamentals of Compressed Air

flyer 4 3 4 2 4

Average Score 3.8 3.0 3.4 3.0 3.6

*Scoring Key: 1 = Not at all; 2 = Somewhat; 3 = Mostly; 4= With certainty

Training Content

AEO and the utilities collaboratively decided training topics during the PWC meetings. All the

stakeholders Cadmus interviewed were generally pleased with the selection of topics, quality of

delivery, and instructors delivering the training sessions.

When the evaluation team asked training participants which topics they found most useful in the

training, respondents most frequently (40%) said the technical information, and about a third (29%) said


33

the application tools, software or analytical methods for energy management. However, many

respondents could not remember specific topics since they took the training more than one year ago.

About three-quarters (74%) of respondents said the instructor or a utility representative discussed the

utility-funded energy-efficiency incentive programs during the training (15% could not remember; 11%

said, according to their memory, this was not discussed).

Of those who recalled discussing utility programs, 65% thought the information helped them

understand how to participate in the programs, while 15% of respondents said it did not, and 20% of

respondents could not remember. Forty-one percent of all respondents said they had participated in at

least one of the following utility-funded energy-efficiency incentive program areas:

Lighting

Audits

Faucet Aerators

Air Audits

Refrigeration

Steam Rebate

HVAC

Motors

Furthermore, 70% of trade allies said the instructor provided guidance on how to promote energy

efficiency to their end-user customers. Of these trade allies, most (71%) thought it was “very useful”

because they thought it was good refresher information and that it was good practical information that

can be easily applied.

Of the 30% of respondents who said they did not receive training on how to promote energy efficiency,

all of them were very interested in receiving this type of guidance in the future.

When asked if there were any topics they would like covered in future EEA training sessions,

respondents mentioned the following areas:8

1. Building efficiency class (15%)

2. Refrigeration (15%)

3. Renewable energy (11%)

4. Energy management tools (11%)

5. Basic air compressor systems (8%)

6. Electricity generation and transmission (4%)

7. Smart metering (4%)

8 Percent of respondents shown. Multiple responses allowed; percentages may not sum to 100%.


34

Furthermore, a few respondents said they would like to receive refresher courses through EEA because

they forgot some of the information learned through the initial in-depth training.

Training Delivery

All of the training sessions were in-person with a qualified instructor assigned by EEA or AMS. Training

sessions ranged anywhere from one to five days long at varying locations across the state.

Several stakeholders Cadmus interviewed thought the training sessions were too long, a likely barrier to

participation among attendees who cannot take that much time away from work. However, when asked

how satisfied they were with the format of the training, 96% of training respondents said they were

either “very satisfied” or “somewhat satisfied.”

When asked about the appropriateness of the level of technical detail provided in the training, most

attendees (77%) said they thought it was just right, 16% thought the training needed more technical

detail, and 8% thought it needed less technical detail. From reviewing the program materials, Cadmus

found that the flyers did not always have clear requisites for level of skill needed to attend the trainings.

Nearly all (96%) respondents thought the instructor who led the training was knowledgeable and

credible about the subject matter (the one respondent accounting for the remaining 4% could not

remember).

Just over half (56%) of respondents recalled the instructor used interactive learning tools such as videos

or digital models (15%), hands-on demonstrations (11%), or team exercises (30%) and all of them

thought this was “very useful” or “somewhat useful” (Figure 14). Nineteen percent of respondents could

not remember if instructors used these types of tools in class, and 15% said there were not interactive

learning tools.

Overall, participants found the training very useful (Figure 14).

Figure 14. Usefulness of Training

10%

90%

4%

15%

81%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Not at alluseful

Not too useful Neutral Somewhatuseful

Very useful

% o

f R

esp

on

de

nts

Interactive learning tools (n=20) Training overall (n=26)


35

Fifty-six percent of respondents said they received pamphlets, brochures, or hand-outs from the

instructor. Fifteen percent could not remember, and 8% said they received a software program or web-

link to a software program.

Satisfaction

When asked how useful they thought the training was overall, 81% said “very useful,” 15% said

“somewhat useful” and one respondent said “not too useful.”

Seventy-eight percent of respondents were “very satisfied” with the training, while 22% were

“somewhat satisfied.” These results align with the high satisfaction rates from the post-training surveys

submitted by training participants upon completion of the course.

Figure 15 shows training participant satisfaction with the program.

Figure 15. Satisfaction with the Training Overall (n=27)

Impacts of Training

When asked how the training impacted their business, respondents most frequently said they were able

to increase their technical knowledge (48%) or increase their awareness of energy efficiency (26%).

Other benefits to participating included saving energy through energy-efficiency actions (15%), receive

educational credits or certifications (15%) and gaining industry contacts (8%).

Overall, participants took several actions related to energy efficiency, including the following most

frequently mentioned actions:9

Talking about energy efficiency with colleagues (41%)

Contractors talking about energy efficiency with end-user customers (22%)

Installing energy-efficient equipment (19%)

Applying for an incentive from a utility-sponsored energy-efficiency program (19%)

Did not take any actions related to energy-efficiency after the training (22%)

9 Percent of respondents shown. Multiple responses allowed; percentages may not sum to 100%.

0% 0% 0%

22%

78%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Not at allsatisfied

Not toosatisfied

Neutral Somewhatsatisfied

Very satisfied

% o

f R

esp

on

de

nts

Satisfaction Rating


36

When comparing contractors to end-users who attended the trainings (Figure 16), contractors most

frequently talked about energy efficiency with their colleagues (40%) and end-user customers (35%).

They also installed energy-efficient equipment at a customer’s facility (20%) or talked with end-user

customers about participating in a utility energy-efficiency program (20%).

C&I end-users who attended the trainings most commonly reported talking about energy efficiency with

colleagues (35%), or applying for a utility incentive for making energy-efficient upgrades at their facility

(29%). C&I end-users that attended the trainings also looked for more information about energy

efficiency (18%), performed an audit or evaluation of their facility/equipment (18%), and installed

energy-efficient equipment (18%).


37

Figure 16. Actions Taken by Participants Since Training

40%

30%

35%

20% 20%

10%

35%

24%

18%

29%

18% 18%

6%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Talked about EEwith colleagues

Nothing [ContractorsOnly] Talked

about EE withcustomers

Installed EE equipment at a

customer’s facility/own

facility

Applied for an incentive from

utility for making energy-efficiency upgrades at own

facility/a customer’s

facility

Looked for moreinformation

about EE

Performed anaudit or

evaluation offacility or

equipment

[ContractorsOnly] Talked

with customersabout

participating in autility EEprogram

Incorporated EEinto sales and

marketingefforts

Adjustedbusiness

operation to runequipment onlywhen needed

% o

f R

esp

on

de

nts

Actions Taken by Participants Since Training

Contractors (n=10)

End-Users (n=17)


38

Of the respondents who said they took action, half (50%) thought the EEA training was “very influential” in getting

them to make those changes, and 35% said EEA training was “somewhat influential.” One respondent said the

EEA training was “not too influential,” and one said it was “not at all influential” in getting them to take the

actions. Figure 17 shows how influential EEA’s training was on participants’ decisions to take these actions.

Figure 17. Influence of EEA Training on Participants’ Actions (n=20)

Barriers

Training participants mentioned several challenges they experience in integrating energy efficiency into their

business practices. Most commonly mentioned were:

Participants do not think energy efficiency is necessary or believe their existing business model works fine

(28%)

Limited interest among their customer base in energy efficiency (12%)

Limited time or too much hassle (12%)

Forty percent of participants thought that continued education on energy efficiency for both businesses and end-

users would be helpful in overcoming these challenges (20%) as well as more rebates to fund energy-efficiency

upgrades (20%). Forty-five percent of participants did not have suggestions for ways to overcome these

challenges (multiple responses allowed).

When the evaluation team asked trade allies specifically about barriers, they most commonly thought that limited

capital was the greatest barrier to making energy-efficiency upgrades among end-user customers (40%), along

with the benefits of energy efficiency not being clear to end-user customers (30%). Note that first cost is the most

common barrier to participation among businesses in most C&I energy-efficiency programs.

To address these challenges, contractors thought end-user customers could benefit from more information on

energy efficiency (30%), more information on rebate programs (10%), and more energy-efficiency resources

(10%).

50%

35%

10%

5% 5%

Very influential

Somewhat influential

Neutral

Not too influential

Not at all influential


39

Aside from training, some respondents had suggestions for other ways EEA can help them integrate energy

efficiency into their facilities and business practices. Most commonly, respondents wanted more updates and

information about energy-efficiency news/classes/new technologies in the form of newsletters and mailings

(15%), more information they [contractors] can pass to end-users about energy efficiency (15%), and case studies

(11%).


40

Conclusions and Recommendations

Cadmus developed the following conclusions and recommendations for program improvements based on the

findings from the evaluation activities previously discussed. Cadmus organized following conclusions and

recommendations by research objective, which apply to both the residential Outreach and Promotion and C&I

Training components of the program unless otherwise specified.

Program Design

Conclusion: The current design of the Outreach and Promotion is meeting the goal of increasing awareness

among residential Arkansas residents.

Conclusion: EEA can improve communication and coordination between the various stakeholders involved in the

Program. Specific problems to communication include unclear and undefined objectives, clarification of target

audiences, inconsistent and ad hoc coordination and promotion of trainings, and little to no data or results

sharing. These factors impact the strategies developed to achieve EEA’s objectives and the success of its activities

in terms of reach and participation.

Conclusion: Due to inconsistent branding and messaging, residential customers may not associate EEA and the

Tighten Up campaign together, thus limiting the program’s ability to build stronger awareness.

Conclusion: There is no explicit branding or content embedded in the training materials or agendas about utility

rebate programs, however, learning about the utility programs is a driving motivator among attendees’ reasons to

sign up.

Conclusion: EEA and the utility energy efficiency programs could be better integrated.

Recommendation 1: Involve all stakeholders (EEA, vendors, the utilities, plus others as necessary) in the

collaborative development of an annual program implementation plan for EEA. This plan should include:

Defined roles, responsibilities and partnerships, including a staff person in-charge of overseeing outreach

among supporting staff and stakeholders

Defined objectives and measureable goals

Key target audiences and strategies to reach those audiences

Branding and messaging strategy, including guidelines for stakeholder use and templates, especially for

promotion of trainings

Plans for keeping the website updated as a usable resource for both residential consumers and the C&I

training component

Outreach strategies and specific activities including protocols for coordination with stakeholders and a

schedule

Training topics and schedule for the year

Plans for tracking progress towards goals, measuring effectiveness, and sharing results with stakeholders

on a regular basis

Meeting and reporting schedule for all stakeholders


41

Recommendation 2: Develop and implement a consistent and comprehensive brand and messaging strategy to

strengthen long term awareness and recognition, and ultimately support EEA’s goal to reach and engage key

audiences.

Recommendation 3: In collaboration with Recommendation 1, solidify partnerships and communications with

utility energy-efficiency programs by formalizing the utilities’ role in developing the training agenda and

content as well as developing the marketing strategy and plan.

Program Performance and Implementation

Conclusion: Even with a limited marketing budget, awareness of EEA or the campaign name, Tighten Up was

approximately 39%.

Conclusion: Despite awareness levels, residential consumers lack familiarity with EEA. With utilities and general

internet searches as consumers’ primary energy-efficiency resource choices, EEA has room to expand reach and

familiarity among Arkansas residential consumers.

Conclusion: The C&I training instructors selected to lead the trainings provided high-quality education and were

knowledgeable about the subject matter.

Conclusion: There are not enough program resources dedicated to outreach for C&I training, resulting in several

challenges to meeting goals.

Conclusion: E-mail is the most effective and preferred way for reaching C&I training participants, and it may be a

cost-efficient way to continue ongoing outreach to targeted markets.

Conclusion: Not all C&I training attendees thought the level of technical detail was appropriate for them, and the

program materials did not always specify the level of technical detail that would be covered in the training.

Recommendation 4: Revisit program budgets to strategically prioritize expenditures that generate the greatest

impact for the program. Based on the revisited budget, move forward with those areas identified as priorities.

Market Effects

Conclusion: Nearly all stakeholders and training participants are satisfied with the trainings and find them useful.

Conclusion: The C&I training is effectively increasing technical knowledge and awareness of energy efficiency

among participants, as well as stimulating participants to take actions such as talking about energy efficiency with

colleagues and customers, installing energy-efficient equipment, and applying for incentives from utility funded

programs.

Conclusion: EEA is missing opportunities for increased and longer-term impacts of the C&I trainings by not

performing post-training follow-up or tracking attendance.


Conclusion: There are missed opportunities to track performance and facilitate future effective evaluations

because not enough information is being tracked.


42

Recommendation 5: Develop a database, including participation data and metrics, which allows AEO to monitor

and maintain information necessary for a comprehensive evaluation and to track performance.

Data collection fields for marketing could include, if available, contact information for hotline callers

including name, phone number, email and mailing address, type of information customer seeks, how

customers heard about EEA, preferred method of communication, and traditional and evaluative metric

results for various marketing tactics.

Data collection fields for training should include first name, last name, title, street address, city, state, zip,

phone number, e-mail, training type, training date, company name, company type10, and a unique

identification number to track participants across trainings and utility energy-efficiency programs.

EEA should work with the utilities to track cross over participation from EEA to energy-efficiency

programs, when relevant.

If not cost prohibitive, collect more feedback from training participants to get better information about

market effects. Perhaps perform a longitudinal study with all C&I training participants to gauge long- and

short-term market effects and to increase participants’ recall rates. The study could include a brief on-site

survey, including questions that rate participant satisfaction and usefulness of the training at the

conclusion of the training11; a second survey a few months later to ask about the changes they have made

to their business practices; and another survey(s) several months later asking about any additional

changes they have made.

10

This information will help EEA understand audiences and align target markets. Specific field options should include: contractor [specify type], end user [specify type: school district, state agency, etc.], utility representative [specify type: utility represented], EEA representative [specify type], other [specify type]

11 There is already a brief survey collected from participants with quantitative and qualitative questions.


43

Areas for Improvement

Additionally, the following are suggested areas for improvement for the EEA effort.

Suggestion 1: Increase and deepen awareness levels by continuing TV and radio outreach channels. Target

media outreach channels and tactics to reach those who are proactively seeking energy-efficiency information

through online sources.

Suggestion 2: Consider targeting more compelling messaging to encourage the “already committed” residential

group that may have started down the path to making energy-efficiency investments beyond the low/no cost

alternatives. Challenge perceptions that homes are already energy efficient and other perceived barriers by

engaging families, whole households, and communities in the Tighten Up challenge. Attitudes and household

characteristics of those that are most likely to engage in energy efficient upgrades include:

Larger homes

Less than 20 years tenure living in their current home

Oklahoma Gas utility customers

May have already made some improvements and received rebates

Are aware that their home could be more energy efficient

Suggestion 3: Consider enriching the C&I training curriculum and delivery methods through these

enhancements:

Add more detail to C&I training outreach materials about desired audience and level of technical detail to

target audiences more effectively.

Offer shorter C&I training sessions for specific target groups (i.e., executives and key decision makers) and

for follow-up refresher courses.

Embed in every course, formal training about how to promote and sell energy efficiency to end-users, to

colleagues, or to key decision makers within businesses. In addition, a separate course specially focused

on this topic may be warranted.

Add content to the trainings on how to sell energy efficiency to end-users, to colleagues, or to key

decision makers within businesses.

Depending on the target market, add in some shorter sessions or other delivery formats, such as

Webinars, that might be better suited for some less in-depth training courses.

Suggestion 4: After each training session, follow up with every attendee to reinforce the training topic and

general efficiency message. EEA staff should take the following steps:

Remind attendees of ways they can take action and implement the concepts learned in the course

Send attendees information about utility programs and ways to participate

Offer quick refresher courses on the topics they already learned about to ensure long-term results

(webinar format could work great for this and reduce administration costs)

Alert participants to other trainings and keep them abreast of all relevant upcoming opportunities

Ask attendees to suggest colleagues or business associates who may also benefit from attending training

sessions


1

7.0 Appendix C: Arkansas Industrial Energy Clearinghouse Report

Arkansas Industrial Energy Clearinghouse

Annual Report January – December 2013

To: Energy Efficiency Arkansas (EEA) Program

Arkansas Economic Development Commission - Energy Office 900 W. Capitol Ave., Suite 400

Little Rock, Arkansas 72201

From: Dr. Darin Nutter

Arkansas Industrial Energy Clearinghouse University of Arkansas

Mechanical Engineering Department 863 West Dickson Street Fayetteville, AR 72701

479-575-4503 [email protected]

Friday, January 31, 2014


2

Introduction and Background The Arkansas Industrial Energy Clearinghouse (or the ‘Clearinghouse’) was originally funded by the Arkansas Energy Office through support of the American Recovery and Reinvestment Act (ARRA) from February 2010 through March 2012. Then for April and May 2012, the Clearinghouse staff and efforts were supported through U.S. Department of Energy’s Industrial Assessment Center funds. Since June 2012, the Clearinghouse has been sponsored by Energy Efficiency Arkansas (EEA), a partnership between the Arkansas Economic Development Commission’s Energy Office and Arkansas’ investor-owned electric and gas utilities and electric cooperatives through a ratepayer funded program approved by the Arkansas Public Service Commission. The primary function of the Clearinghouse is to support and promote energy efficiency within Arkansas’s industrial and manufacturing sector. The Clearinghouse is housed at the University of Arkansas’s Mechanical Engineering Department located in Fayetteville, Arkansas. The Clearinghouse includes an on-line website for contact information and links to resources, a full time staff engineer, part-time engineering students, and a program director (Dr. Darin Nutter). Typical Clearinghouse tasks and activities are to: 1. Assist companies in starting an energy management program, 2. Provide independent review of energy projects, 3. Support utility Energy Efficiency Program incentives, 4. Provide an understanding of industrial system(s) energy performance, 5. Recommending energy and cost-savings opportunities, 6. Providing technical resources, such as papers, publications, and loaning instruments, 7. Maintaining up-to-date knowledge of financial incentives for energy conservation, 8. Provide on-site plant assessments and system energy/power measurements, 9. Recommend system specific experts, as needed, such as DOE system specialists, and 10. Maintain a website with Arkansas specific information and tools. Summary of Activities during Progress Period As shown in Table 1 below, during the period of January 2013 through December 2013, the Clearinghouse served 58 different companies in the state of Arkansas. In addition to supporting the manufacturers directly, the Clearinghouse indirectly supported these companies through interactions with their utility companies and their energy efficiency programs.

Table 1. Company Interactions (2014 and Clearinghouse Life-to-Date)

Time Contacts Companies Web hits Web visitors

Jan 2013 – Dec. 2013* 494 89 11,299 5,651

June 2010 –Dec. 2013 (LTD) 2,910 388 36,472 14,778 ______________________________________________________________________________________

*58 of the 89 companies were Arkansas industrial manufacturers.


3

Table 2 below provides as summary of various interactions with Arkansas manufacturers during the

2013 calendar year. The Clearinghouse recommended the implementation of numerous energy

efficiency measures that will reduce the use of electricity and natural gas, as well as utility costs. These

recommendations range from written engineering reports, which provide detailed calculations of

potential savings, to written lists of recommendations based on a one-day site visit. During 2013, the

Clearinghouse made 56 site visits to 28 manufactures and made recommendations for these and other

manufacturers. The number site visits was up compared to any previous year. Only two visits with

manufacturers did not result in recommendations. It is common for companies to have their own

unique selection criteria as to whether a measure is implemented, so various implementation rates are

seen. In total for the project period, the recommendations had an estimated implementation cost of

$2.2 million, annual cost savings of $1.2 million, and annual energy savings (site basis) of 150,000

MMBtu.

Table 2. Summary of 2013 Interactions with Arkansas Manufacturers

Entity Companies Interactions Site Visits

Entergy 25 92 19

Swepco 19 103 16

OG&E 6 47 10

Empire District

1 15 6

Co-ops 4 32 4

Municipals 3 22 1

Subtotals: 58 311 56

CenterPoint 33 132 21

SourceGas 19 132 25

AOG 6 47 10

Subtotals: 58 311 56


4

Brief discussion of noted work during period:

The Clearinghouse efforts for 2013 again had wide coverage of services across the state. The number of site visits to manufactures increased significantly, as did the quality of recommendations. This was largely due to our increased in experience and ability to directly serve Arkansas manufactures with recommendations they are more likely to implement. The Clearinghouse also had an increase in our instrumentation loan program (129 instruments loans), which positively impacts both manufactures (loaned to 13 companies, including 49 data logger/CT sets, 8 pressure loggers 4 thermal image cameras, anemometers, pitot tubes, ultrasonic leak detectors, light meters, data logger/temperature/RH sets, power quality analyzers, and flow meters); utility subcontractor (loaned 5 times, including date loggers and thermal imaging camera); equipment vendors; and UA. More specifically, the year included the promotion and completion of energy efficiency efforts in a number of ways. Selected examples include:

1. completion of a whole-system compressed air analysis for an Arkansas sanitary paper products manufacturer. The recommendations were implemented with very positive feedback from the company, Cadmus (the utility incentive program’s independent verification contractor), and others. Results from this study will be published in May 2014 at the Industrial Energy Technology Conference (IETC), with paper titled “Compressed air system analysis and retrofit for energy savings – an industrial case study”;

2. completion via follow-up with southern Arkansas company. Per the request of the company, the Clearinghouse re-visited the plant to provided details on previous energy savings recommendations. Implementation included lights, compressed air, and steam system.

3. completion of a significant effort with a poultry processing plant. This was a detailed energy analysis for the processing plant. The Clearinghouse portion of the project included a comprehensive energy efficiency study with analysis and recommendations with the potential to reduce the plant’s utility costs by over 20%;

4. completion of a new Clearinghouse website with the addition of a combined heat and power (CHP) section;

5. Dr. Nutter’s participation as a committee member of the National Governor’s Association (NGA) Arkansas Policy Academy to “enhance manufacturing through energy efficiency and combined heat and


5

power”. His involvement also included being a presenter at the Arkansas Industrial Energy Efficiency Finance Forum;

6. a study of combined heat and power (CHP) system application was started and is on-going. This UA study’s working title is “Modeling the benefit of a joined combined heat and power (CHP) and thermal energy storage (TEM) system: an industrial scenario analysis for energy and greenhouse gas (GHG) minimization”.


Documents

EEA Annual ReportX(1)S(s5ibn0ff5eln2o55oizkej55))/EEInfo... · Increased Staffing –to increase grassroots marketing efforts, EEA added an EEA Facilitator to reach the underserved