Market Progress Evalu ation Report Efficient Building ... · Market Progress Evalu ation Report Efficient Building Practices Initiative,No.2 ... Response to the Website Usability

529 SW Third Avenue, Suite 600Portland, Oregon 97204telephone: 503.827.8416 • 800.411.0834fax: 503.827.8437

M a r k e t P r o g r e s s E v a l u a t i o n R e p o r t

Efficient Building Practices Initiative, No. 2

prepared by

Research Into ActionDethman & Tangora

report #E01-085

July 2001

Final Report

Market Progress Evaluation Report #1

EFFICIENT BUILDING PRACTICES INITIATIVE

Volume 1: Report

Funded By:

Submitted To:

Jane Gordon, Ph.D.

Northwest Energy Efficiency Alliance

Prepared By:

DETHMAN & TANGORA h

Linda Dethman Dethman & Tangora, LLC

Jane S. Peters, Ph.D.

Marjorie McRae, Ph.D. Research Into Action, Inc.

July 2001

EBPI – MPER #1 DETHMAN & TANGORA h

ACKNOWLEDGEMENTS

We wish to thank the Northwest Energy Efficiency Alliance for funding this evaluation effort and for the support and cooperation of Alliance staff, especially our project manager, Jane Gordon. In addition, we thank the Efficient Building Practices Initiative Steering Committee, Ecotope, Inc., Construct Inc., Cole & Weber, Inc., Hypercerulean, Inc., and Gilmore Research, Inc. for their contributions to this evaluation research. Finally, we thank all the respondents to our surveys, those participating in the website usability testing, and the focus groups for their time, cooperation, and helpful insights.

Acknowledgements

DETHMAN & TANGORA h EBPI – MPER #1

EBPI – MPER #1 DETHMAN & TANGORA h i

TABLE OF CONTENTS

GLOSSARY...............................................................................................................................GL-1

EXECUTIVE SUMMARY..............................................................................................................ES-1

INTRODUCTION .............................................................................................................ES-1

EVALUATION ACTIVITIES ............................................................................................ES-1

WHAT DID EBPI SET OUT TO DO?................................................................................ES-2

WHAT CAN WE CONCLUDE ABOUT EBPI’S ACCOMPLISHMENTS? ..........................ES-3

WHAT ARE THE MAJOR LESSONS OF EBPI? ..............................................................ES-5 The Regional Public Information Program (RPIP) ................................................ES-5 Code Support, State Special Projects, Seed Funding .............................................ES-8

WHERE SHOULD THE COMMERCIAL BUILDING INITIATIVE GO FROM HERE? ..ES-10 Toward a New Theory ........................................................................................ES-11 Theory Recommendations...................................................................................ES-12 Both Segments and Intersections Matter ............................................................ES-13

Segmenting the Market ......................................................................................ES-13 Recommendations – Segmentation and Integration ............................................ES-14

WHAT MARKET CHANGE INDICATORS SHOULD WE LOOK FOR? .........................ES-15

CHAPTER 1: THE EFFICIENT BUILDING PRACTICES INITIATIVE.....................................................1

INTRODUCTION ..................................................................................................................1

HISTORY OF EBPI...............................................................................................................1

EBPI PROGRAM ELEMENTS ..............................................................................................3 Regional Public Information Program (RPIP) ...........................................................3 New Construction Baseline........................................................................................8 Energy Code Support.................................................................................................9 State Special Projects Program ..................................................................................9 Seed Funding .......................................................................................................... 10 Transition Funding for Infrastructure ..................................................................... 10 Code Advisory Committee ....................................................................................... 11

KEY FINDINGS FROM THE BASELINE REPORT ............................................................ 11 Commercial Employee Interviews............................................................................ 11 Architect and Developer Interviews ......................................................................... 13 Conclusions and Issues............................................................................................ 14

Table of Contents

DETHMAN & TANGORA h EBPI – MPER #1 ii

CHAPTER 2: THE EBPI EVALUATION APPROACH.......................................................................15

THE EBPI LOGIC MODEL ................................................................................................. 15

EVALUATION COMPONENTS FOR EBPI......................................................................... 17

CHALLENGES TO EVALUATING EBPI............................................................................ 17

CHAPTER 3: DEMAND-SIDE RESPONSE TO THE RPIP .................................................................19

BACKGROUND AND PURPOSE........................................................................................ 19

METHODOLOGY................................................................................................................ 19 Sampling Plan ......................................................................................................... 19 Methodology for Assessing Awareness..................................................................... 20

SAMPLE CHARACTERISTICS........................................................................................... 21

AWARENESS OF THE BETTERBRICKS.COM AD CAMPAIGN ........................................ 24 Awareness of the Ad Campaign and of the betterbricks.com Name........................... 24 Awareness by Advertising Media............................................................................. 25 Awareness by Respondent Characteristics............................................................... 26 Interest Generated by Campaign ............................................................................. 26 Visits to Website...................................................................................................... 27 Characteristics of betterbricks.com Visitors.............................................................. 28 Response to the Website .......................................................................................... 29

PHYSICAL WORKSPACE AND EMPLOYEE PRODUCTIVITY ......................................... 29

INFORMATION SOURCES ................................................................................................ 34

USAGE PATTERNS OF PUBLICATIONS AND THE INTERNET ...................................... 35

SUMMARY OF AD CAMPAIGN AWARENESS .................................................................. 37

CHAPTER 4: ARCHITECT ASSESSMENTS FOR THE RPIP...............................................................39

INTRODUCTION ................................................................................................................ 39

METHODOLOGY................................................................................................................ 39 Survey Design ......................................................................................................... 39

Sample .................................................................................................................... 40

FINDINGS.......................................................................................................................... 41 Client Interest in Energy Efficient Design Solutions................................................ 41 Awareness of the RPIP betterbricks.com Campaign ................................................. 44

SUMMARY OF FOLLOW-UP WITH ARCHITECTS ........................................................... 46

CHAPTER 5: BETTERBRICKS.COM ADVERTISING FOCUS GROUPS...........................................47

INTRODUCTION AND GOALS .......................................................................................... 47

METHODS .......................................................................................................................... 47

Table of Contents

EBPI – MPER #1 DETHMAN & TANGORA h I I I

KEY FINDINGS AND IMPLICATIONS.............................................................................. 48 Roles and Perceptions About Workspace Design Issues and Trends ......................... 49 Reactions To Television Advertising ........................................................................ 50 Reactions To Print Advertising ................................................................................ 51 Characterization of the betterbricks.com Brand Identity .......................................... 52

Information Sources ................................................................................................ 52

RECOMMENDATIONS ...................................................................................................... 53

CHAPTER 6: BETTERBRICKS.COM WEBSITE USABILITY TESTING AND USE.................................55

INTRODUCTION ................................................................................................................ 55

RPIP’S BETTERBRICKS.COM WEBSITE USABILITY TEST ............................................ 55 Introduction and Goals ............................................................................................ 55 Methods .................................................................................................................. 56 Overall Usability Results......................................................................................... 57 Key Findings and Recommendations ....................................................................... 58 Response to the Website Usability Test Recommendations ...................................... 64

WEBSITE USAGE .............................................................................................................. 65 Introduction ............................................................................................................ 65 Advertising Reach and Website Hits........................................................................ 65 Website Hits and Target Market Size...................................................................... 67 Return Visitors Survey ............................................................................................ 69

BETTERBRICKS.COM ADVISORS’ EXPERIENCE............................................................ 70 Advisor Service Assessment..................................................................................... 76

SUMMARY OF WEBSITE RESEARCH .............................................................................. 77

CHAPTER 7: EVALUATION OF ENERGY CODE SUPPORT...........................................................79

INTRODUCTION ................................................................................................................ 79

EVALUATION PURPOSES AND METHODS ..................................................................... 79

MEETING FUNDING GOALS ............................................................................................ 80 Overview of Goals and Activities ............................................................................. 80 Description of Goals and Activities by State ............................................................ 82

VIEWS OF ALLIANCE PROJECT MANAGEMENT ........................................................... 88

FUTURE CODE OUTLOOK................................................................................................ 89 Idaho....................................................................................................................... 90 Oregon .................................................................................................................... 91 Washington ............................................................................................................. 92

CONCLUSIONS.................................................................................................................. 93

Table of Contents

DETHMAN & TANGORA h EBPI – MPER #1 iv

CHAPTER 8: STATE SPECIAL PROJECTS.......................................................................................95

INTRODUCTION AND METHODS .................................................................................... 95

MEETING ALLIANCE GOALS........................................................................................... 96 Meeting State Special Projects Goals ....................................................................... 96

PROJECT DESCRIPTIONS AND KEY ACCOMPLISHMENTS.......................................... 99 Green Building Baseline Study: $40,000 in Funding ............................................... 99 Demand Controlled Ventilation (DCV): $130,000 in Funding ................................ 100

Idaho Energy Code Support Project: $77,500 in Funding ...................................... 101 GEMSTAR: $60,000 in Funding ........................................................................... 102 Special Plans Examiner/Inspector Certification Project (SPE/I): $33,694 in

Funding 102 Oregon Energy Code Website: $58,000 in Funding................................................ 103 Shorebank Pacific: $5,000 in Funding ................................................................... 103 Prescriptive Residential Duct Code: $25,000 in Funding....................................... 104

VIEWS OF ALLIANCE PROJECT MANAGEMENT ......................................................... 105

ADVICE TO THE ALLIANCE........................................................................................... 106

CONCLUSIONS................................................................................................................ 107

CHAPTER 9: SEED FUNDING AND THE NEW CONSTRUCTION BASELINE................................109

INTRODUCTION AND METHODS .................................................................................. 109

PROGRESS FOR FUTURE AT WORK.............................................................................. 109

PROGRESS FOR BREWERY BLOCKS PROJECT ............................................................ 110

NEW BUILDINGS INSTITUTE/DESIGN GUIDELINES .................................................. 112

SEED FUNDING CONCLUSIONS ................................................................................... 113

NEW CONSTRUCTION BASELINE SURVEY.................................................................. 114

CHAPTER 10: EBPI PROGRESS INDICATORS.............................................................................117

INTRODUCTION AND INDICATORS.............................................................................. 117 1. Demand for Professionals with Energy Efficiency Expertise Increases .............. 117 2. Awareness of Productivity and How It Relates to Energy Efficiency

Increases Among Nonresidential Demand-Side Actors ...................................... 119 3. Availability of Energy Efficiency Professionals Increases .................................. 119 4. Energy Efficiency Activities in Buildings Depend Less on Alliance Funding

and Move Toward Sustainability....................................................................... 120 5. Customer Demand for Energy-Efficiency Services Stimulates Supply-Side

Professionals to Provide Solutions/Services....................................................... 120 6. Energy Efficiency Increases Because Supply-Side Professionals, Responding

to Greater Demand, Offer Consumers More Energy Efficient Solutions............. 121 7. Energy Codes are Maintained and Remain Viable............................................. 121

Table of Contents

EBPI – MPER #1 DETHMAN & TANGORA h V

8. Alliance Stakeholders See EBPI as a Regional Effort ........................................ 121 9. EBPI Has a Promising Strategy and There Is Evidence That There Is a

Reason to Continue, As Is or Modified, Based on Evaluation Findings .............. 122

CONCLUSIONS................................................................................................................ 122

CHAPTER 11: CONCLUSIONS AND LESSONS FOR EBPI ..........................................................123

INTRODUCTION .............................................................................................................. 123

WHAT DID EBPI SET OUT TO DO?................................................................................. 124

WHAT CAN WE CONCLUDE ABOUT EBPI’S ACCOMPLISHMENTS? ........................... 125

WHAT ARE THE MAJOR LESSONS OF EBPI? ............................................................... 127 The Regional Public Information Program (RPIP) ................................................. 127 Code Support, State Special Projects, Seed Funding .............................................. 131 The New Construction Baselines ........................................................................... 134

CHAPTER 12: SYNTHESIS ............................................................................................................135

INTRODUCTION .............................................................................................................. 135

HOW CAN COMMERCIAL BUILDING INITIATIVES CREATE MARKET CHANGES? .. 135 Theory Underlying Market Change....................................................................... 135 EBPI Theories In A Nutshell................................................................................. 136 EBPI Theories Revisited........................................................................................ 137 Recommendations ................................................................................................. 140

WHAT TARGET MARKETS SHOULD BE PURSUED WITH WHAT STRATEGIES? ....... 140 Introduction .......................................................................................................... 140 Segmenting the Market ......................................................................................... 141 Recommendations ................................................................................................. 141

WHAT MARKET CHANGE INDICATORS SHOULD WE LOOK FOR? ............................ 146

REFERENCES FOR CHAPTER 12 ................................................................................... 146

Table of Contents

DETHMAN & TANGORA h EBPI – MPER #1 v i

APPENDICES

APPENDIX A: BETTERBRICKS.COM QUESTIONNAIRE............................................................. A-1

APPENDIX B: ARCHITECTURE + ENERGY (A+E) 2001 ARCHITECT INTERVIEW GUIDE...........B-1

APPENDIX C: DRAFT FOCUS GROUP REPORT FOR RPIP’S BETTERBRICKS.COM CAMPAIGN

EXECUTIVE SUMMARY................................................................................................... C-I Introduction and Goals ........................................................................................... C-I Methods ................................................................................................................C-II Key Findings and recommendations .....................................................................C-III

CHAPTER 1 – INTRODUCTION .......................................................................................C-1 The Efficient Buildings Practices Initiative and the Regional Public Information Program .................................................................................................................C-1 The betterbricks.com Information Campaign...........................................................C-1 Focus Group Goals .................................................................................................C-2 Focus Group Methods .............................................................................................C-3

CHAPTER 2 – PERCEPTIONS ABOUT WORKSPACE DESIGN ISSUES, TRENDS, AND ROLES ................................................................................................................C-5

Mover and Non -Mover Groups ................................................................................C-5

CHAPTER 3 – REACTIONS TO TELEVISION ADVERTISING.......................................C-11

CHAPTER 4 – REACTIONS TO PRINT ADVERTISING..................................................C-15

CHAPTER 5 – THE BETTERBRICKS.COM BRAND .......................................................C-17 Adjective Choice...................................................................................................C-17 What will you find at website? ..............................................................................C-18 Description of betterbricks.com .............................................................................C-18

CHAPTER 5 – INFORMATION SOURCES AND FINAL ADVICE ...................................C-21 Information Sources .............................................................................................C-21

Final Advice.........................................................................................................C-21

APPENDIX: FOCUS GROUP MATERIALS ....................................................................C-23 Betterbricks.com Business Influentials / Decision-Makers Focus Group Discussion Guide ...................................................................................................................C-25 Betterbricks.com Real Estate Developers / Professionals Focus Group Discussion Guide ...................................................................................................................C-33

Table of Contents

EBPI – MPER #1 DETHMAN & TANGORA h VII

APPENDIX D: DRAFT BETTERBRICKS.COM WEBSITE USABILITY REPORT

TABLE OF CONTENTS.................................................................................................... D-3

EXECUTIVE SUMMARY...................................................................................................D-I Introduction and Goals ...........................................................................................D-I Methods .................................................................................................................D-I Key Findings .........................................................................................................D-II

CHAPTER ONE – INTRODUCTION, PURPOSE, AND METHODS.................................. D-1

Methods ................................................................................................................ D-1 Organization of the Report .................................................................................... D-3

CHAPTER TWO – USABIL ITY ISSUES AND RECOMMENDATIONS ............................ D-5 Communication of Web Site Purpose ..................................................................... D-5 Organizational Issues ............................................................................................ D-5 Navigational Issues............................................................................................... D-7 Content Issues....................................................................................................... D-8 Stylistic Issues ...................................................................................................... D-8 Advisor Area Issues ............................................................................................... D-9 Field Study Issues ................................................................................................D-10

CHAPTER THREE – RESULTS OF TASKS AND ANSWERS TO QUESTIONS ..............D-13

APPENDICES Appendix A: Table of Usability Problems ..............................................................D-21 Appendix B: Participant Comments .....................................................................D-31 Appendix C: Usability Test Procedures ................................................................D-37 Appendix D: Background on Human Computer Interfaces ....................................D-43

APPENDIX E: FINAL EBPI CODES SUPPORT PROCESS EVALUATION INTERVIEW GUIDE ......E-1

APPENDIX F: STATE SPECIAL PROJECTS PROCESS EVALUATION INTERVIEW GUIDE............ F-1

APPENDIX G: FINAL BASELINE EVALUATION REPORT EBPI – EXECUTIVE SUMMARY..........G-1

BACKGROUND ................................................................................................................ G-1

KEY FINDINGS................................................................................................................ G-2 Baseline Assessment Of Commercial Workers ....................................................... G-2 Baseline Assessment of Commercial Architects and Developers............................. G-4

CONCLUSIONS AND ISSU ES ......................................................................................... G-6 1. The BetterBricks.com message is competing with other messages..................... G-6 2. BetterBricks.com messages may not currently attract developers. .................... G-6 3. Primary target audiences prefer news and prime time TV to sports. ................. G-7 4. Preferences for information sources may vary by target audience. .................... G-7

Table of Contents

DETHMAN & TANGORA h EBPI – MPER #1 viii

EBPI – MPER #1 DETHMAN & TANGORA h GL - 1

GLOSSARY

Table G-1 lists and describes acronyms and abbreviations used throughout this report. They are listed in the order they are presented.

TableG-1: List of Terms Used in This Report

TERM DEFINITION

Alliance Northwest Energy Efficiency Alliance – a non-profit group of electric utilities, state governments, public interest groups, and industry representatives committed to bringing affordable, energy efficient products and services to the marketplace. The Alliance is the sponsor of EBPI.

EBPI Efficient Building Practices Initiative – the Northwest Energy Efficiency Alliance program being evaluated in this Market Progress Evaluation Report. As initially conceived, EBPI included these elements: the Regional Public Information Program (RPIP), the New Construction Baseline, Energy Code Support, State Special Projects, Seed Funding, Transition Funding for Existing Infrastructure, and the Code Advisory Group.

RPIP Regional Public Information Program – the public information program of EBPI.

betterbricks.com betterbricks.com – the initial public information campaign (including advertising, public relations, marketing, a website, and other services) launched under RPIP. The campaign’s goal is to inform and persuade commercial workers and architects and developers about the important effects of energy-related workspace design elements (such as lighting and temperature control) on worker productivity.

Glossary

DETHMAN & TANGORA h EBPI – MPER #1 GL - 2

EBPI – MPER #1 DETHMAN & TANGORA h ES - 1

EXECUTIVE SUMMARY

INTRODUCTION

The Northwest Energy Efficiency Alliance (the Alliance) is a non-profit group of electric utilities, state governments, public interest groups and industry representatives committed to bringing affordable, energy efficient products and services to the marketplace. The Alliance’s Efficient Building Practices Initiative (EBPI) couples a Regional Public Information Program (RPIP) with several building code support programs.

The Alliance hopes this pairing of consumer demand and codes infrastructure enhancements will ensure greater compliance with current energy codes and propel voluntary “standard practice” beyond them. The $6.5 million initiative was launched in 1998, with the majority of effort taking place in 1999 and 2000.

EVALUATION ACTIVITIES

The evaluation began in fall of 1999, and was structured to provide both real time evaluation results of the RPIP effort and an end-of-contract-term assessment of all EBPI efforts. The evaluation, therefore, included these activities:

Ø A baseline and follow-up survey with the target markets;

Ø A usability test of the RPIP campaign website;

Ø Focus groups assessing response to the RPIP media efforts;

Ø Interviews with leads for the diverse code-related projects funded under EBPI;

Ø Interviews with Alliance staff and EBPI Steering Committee members; and

Ø Analysis of data collected at the website and through use of the website advisory service.

At this stage of EBPI we have conducted a variety of research activities and have assembled information on each of its components. Because of the many areas of the

Executive Summary

DETHMAN & TANGORA h EBPI – MPER #1 ES - 2

building market that EBPI seeks to influence, we ask the reader to keep the following considerations in mind:

Ø EBPI is a complex program with ambitious goals. Compared to the context in which it is operating and that it is trying to influence – the market for constructing and renovating commercial buildings and the advancement of energy codes – EBPI has had a short life and definitive results should not be expected at this time.

Ø The short-term outcomes specified in EBPI’s logic model (see Chapter 10) have few specifically measurable activities or results attached to them. This means we had few benchmarks against which to compare performance, and this makes it challenging to determine whether the EBPI has worked or not.

Ø This evaluation does not, and was never intended to equally address all components of EBPI. Some aspects of a typical evaluation, such as assessing the program development and decision-making process, were outside the evaluation scope. Furthermore, evaluation activities and priorities changed according to the Alliance's perception of information requirements, especially as the RPIP was launched, and as the EBPI effort evolved during 2000.

Still, we believe examining the short life of EBPI yields very useful conclusions and lessons as the Alliance plans new strategies to address the commercial buildings market (see Chapters 10, 11, and 12 for a fuller discussion). To frame this inquiry, we will address the following questions:

Ø What did EBPI set out to do?

Ø What can we conclude about EBPI’s accomplishments?

Ø What are the major lessons from EBPI?

Ø Where should the commercial building initiative go from here?

WHAT DID EBPI SET OUT TO DO?

To address this question, we need to return to the two goals underlying the design of the initiative. This is an appropriate time to revisit these goals to see if they remain accurate and adequate for describing what EBPI set out to do. EBPI’s goals are:

Executive Summary


Ø To increase the consumer demand among commercial building users and decision-makers for highly energy-efficient buildings over a twenty-year period.

Ø To enhance the capability of code-related institutions so they can meet increased consumer demand for efficient buildings.

The first goal was largely served by the RPIP betterbricks.com effort. The theory was that if one can attract a mass audience, through advertising and marketing, to the betterbricks.com website, and at the website stimulate visitors to be interested in the product (better buildings), then the mass market will eventually demand that product and more products and services will become available to respond to the demand.

Code Support and State Special Projects activities served the second goal. While these efforts were successful in many ways, few of their activities were aimed directly at meeting increased “consumer demand for efficient buildings,” except in the broad sense of helping to ensure that high-efficiency buildings can be built under the codes.

Over the course of this evaluation we have discovered that the two longstanding EBPI goals do not accurately reflect all of EBPI’s activities nor do they adequately identify all the actors that must be reached if the commercial building market is to be transformed. EBPI’s actual and intended efforts to reach design professionals, and to foster a better “handshake” between those demanding more efficient buildings and energy-efficiency professionals, are not reflected in these goals. And, the many other critical actors involved in the commercial development and building process, such as developers, financial backers, real estate professionals, contractors, and government agencies, are not covered under these goals.

WHAT CAN WE CONCLUDE ABOUT EBPI’S ACCOMPLISHMENTS?

If we boil down the desired short-term outcomes from EBPI’s logic model, we believe there are five questions that need to be addressed when summarizing EBPI’s accomplishments. These five questions, and our findings vis-à-vis these questions, are provided below:

Executive Summary


1. Did EBPI increase awareness among demand-side audiences of the connection between productivity and energy efficiency opportunities in commercial buildings?

About 4% of influential employees and decision-makers in commercial businesses became aware of the betterbricks.com campaign. One finding relating productivity to energy efficiency can be attributed to program efforts: those who were aware of the campaign were significantly more likely to mention comfortable temperature as important to productivity.

2. Did EBPI foster more, and more able, supply-side energy efficiency professionals?

There is no baseline data for energy efficiency professionals against which to measure a change, and there were no specific efforts outside of the Brewery Blocks project to stimulate an increase in energy efficiency professionals. EBPI had intended to conduct an advocacy program for designers, architects, and engineers to increase their attention to energy efficiency, but it was not launched.

The betterbricks.com campaign did reach some commercial design professionals, and may have influenced a few to pay more attention to efficiency issues. In a follow-up survey with architects in the region, about 10% said they were aware of betterbricks.com, which was higher than for the primary targets of building users and decision-makers(4%). A few of the aware architects reported they planned to use betterbricks.com resources for their own work.

3. Did EBPI help maintain/enhance energy codes in the region?

Yes. With the help of Alliance funding, code staff continued to maintain the energy codes in Idaho, Oregon, and Washington. Code revisions occurred in Oregon and Washington. Idaho is in the process of adopting the International Building Code; this code includes a more stringent energy code that must be adopted by those who currently have energy codes and by any jurisdictions adopting the IBC.

4. Did EBPI foster self-sufficiency or sustainability in the marketplace (i.e., market transformation)?

Very little effort was exerted in this direction. The Brewery Blocks project successfully involved a private-sector energy and environmental specialist in a large commercial development. This specialist is developing a tenant “users” manual to

Executive Summary


help tenants maintain the project’s energy and environmental improvements. The design approach and manual may become a model for other commercial projects.

5. Will EBPI save energy?

Yes. There are potential energy savings associated with many of the components in EBPI: Code Support, State Special Projects, the projects coming to the Advisory Service at the website, support for the specialist on the Brewery Blocks project, the development and distribution of energy efficiency Design Guidelines, and support for the Future@Work project. All of these activities are likely to result in energy savings, at least over the long-term that would not have occurred without EBPI.

WHAT ARE THE MAJOR LESSONS OF EBPI?

This section summarizes the important, program-specific, insights we’ve gained from evaluating EBPI elements to date. A discussion of larger, crosscutting insights that the Alliance should consider as it plans its strategies for future commercial sector initiatives begins in the section titled “Where Should the Commercial Building Initiative Go From Here?” in this Executive Summary and is explored more thoroughly in Chapter 12 of this report.

The Regional Public Information Program (RPIP)

The RPIP, as embodied in the betterbricks.com campaign, was the centerpiece of EBPI. The majority of program and evaluation resources went to developing, implementing, and examining various aspects of that campaign – for instance, the effects of television and other advertising, the website design and use, and the use of the Help Desk.

Insights from RPIP portion of EBPI are discussed below. Most are based on findings from more than one evaluation activity. (For more detailed findings from individual evaluation activities, please see the Baseline Report, as well as individual chapters of this report and its appendices.)

Executive Summary


1. Embarking on a major mass media and marketing campaign without testing the advertising approaches with the target audiences likely resulted in a less effective campaign.

One of the most important lessons from the RPIP is that mass media approaches and messages need to be tested with target audiences before launch. Substantial work was done to develop the message and to understand what might work with the target markets. Much of this research confirmed previous studies about the need to focus on non-energy benefits, and which non-energy benefits might be a good hook for the markets. However, the translation of this foundational research into RPIP advertising was not tested.

Evaluation findings showed that audiences targeted for the campaign often did not understand the messages being conveyed through the ads, weren’t sure about their intent, and in some cases responded negatively. More specifically, findings revealed:

Ø “Productivity” is a powerful idea but target audiences did not link it easily to energy efficiency.

Ø Relying on productivity as the one link to energy efficiency limited the strength of the campaign.

Ø Indirect approaches (i.e., no obvious connection of the ads to energy efficiency or to the Alliance) did not appeal to audiences.

2. General commercial employees are not likely to influence building space decisions and create demand for more efficient buildings. Business decision-makers and members of the design, development, and financial communities are smaller target audiences that likely offer better leverage points.

The theory behind the betterbricks.com mass media campaign was that general employees (along with influential employees and decision-makers) could be rallied to the cause of better, more productive workspaces, which in turn would foster certain activities related to energy efficiency. The usability test, focus group results, and tracking of project requests through the Advisor Service clearly show general commercial employees do not usually influence building space decisions.

On the other hand, supply-side service providers (e.g., architects, contractors) and business decision-makers did bring projects to the Advisor Service. Website visits and focus groups with business decision-makers revealed that those anticipating

Executive Summary


workspace changes were even more likely to be interested in betterbricks.com. The focus group with developers and real estate professionals showed that this community was likely to be crucial to building decisions, and other research shows those who provide capital also make key decisions. These findings all suggest a set of specialized audiences that need to be targeted as change agents.

3. Given the smaller size and predilections of key audiences (business decision-makers and those in the design and development communities), television advertising is probably not the most effective outreach mechanism. Targeted print ads and stories, as well as more personal outreach mechanisms (e.g., peer referral) are likely to have greater effect.

According to results of the follow-up surveys, television advertising did reach a small percentage of decision-makers, influential staff, and architects. The focus group with developers showed they all had noticed the ads. This suggests that increased and improved advertising would reach more of these audiences, which would, in turn, prompt more visits to the website.

But, is television the most effective way to reach and motivate these small audiences, given the shotgun spray of such advertising? Probably not – television advertising is an expensive way to reach such small audiences. In addition, various findings suggest that these target groups are more likely influenced by their peers, by more person-to-person marketing, and by information that appears in sources particular to their industries.

4. Betterbricks.com has drawing power, but involvement was limited. Key audiences voiced interest in betterbricks.com resources and in partnering arrangements.

We found the respondents in the usability test and focus groups (other than general employees) to be interested in betterbricks.com. Interest was stronger among building users if they were contemplating new space. Interest appeared strongest among designers, architects, and developers, especially if there were opportunities for partnerships with the brand’s service. The follow-up survey with influential employees and decision-makers found that awareness of betterbricks.com was stronger if they had also participated in the baseline survey and thus had been alerted to a potential campaign (6% of “repeat” compared to 2% of “new” respondents). Overall, only 4% of the entire sample was aware of the campaign.

Executive Summary


Thus, the involvement with the betterbricks.com brand is limited after this first year of effort.

5. To take action, betterbricks.com website users needed more in-depth information of the right kind, and probably other types of help, than the site supplied. Lack of “next step” information and technical assistance hurt program progress.

Most usability test subjects wanted more depth than what was on the site. Architects in the follow-up survey noted that the website materials were good to show clients ideas, but did not really help them design or sell a project. There are many directions that can be pursued to increase the depth of help, such as providing more information on: technical topics, budget impacts, contacts for assistance, and the strengths and weaknesses of each of the solutions presented on the website. The materials should be tested with target audiences before final production or site launching to ensure that they meet audience needs.

Code Support, State Special Projects, Seed Funding

The other major arm of EBPI supported code-related activities, from direct support of agencies developing and maintaining energy codes to projects that advanced new code-related technologies. Conclusions from these efforts are reported below.

Code Support

1. Alliance code support has worked effectively to maintain and advance energy codes in Idaho, Washington, and Oregon. Permanent, sustainable funding through state sources for code support activities is a difficult goal to achieve and ongoing outside support will likely be needed.

Without the funding in Idaho, Washington, and Oregon, energy code support and maintenance would have been, at the least, impaired and, at the most, non-existent. Funding for code support appears to ebb and flow both at state and national levels, and agencies did not report much progress toward being self-sufficient. Over time, the lack of code support will result in lower code compliance (i.e., through less well-trained code officials) and less code development.

The results also show the states leveraged the money well beyond its intent of just retaining staff with institutional memory. In Idaho, Alliance funding helped with the preliminary adoption of the IBC; in Oregon, it sponsored innovative solutions,

Executive Summary


such as web access to materials and forms; and in Washington it contributed to 50 to 60 specific code improvements (although a few key changes did not occur).

2. The Alliance would benefit by having a stronger presence and involvement in code issues and policy throughout the region, although it would need to determine its appropriate role.

A strong Alliance role would be particularly helpful in terms of supporting code changes that the states are trying to achieve, and in helping to resolve code “turf” wars. The code agencies would like to see the Alliance provide more technical expertise and higher-level policy and political support. A greater Alliance presence would not only keep it in better touch with code progress or threats, it would also be a credit to the organization.

State Special Projects and Seed Funding

1. State Special Projects and the Brewery Blocks project have achieved specific, valuable – but often invisible – accomplishments.

The State Special Projects have met the Alliance goals of providing innovative code support, fostering voluntary actions beyond code, and building partnerships in the building industry. The Brewery Blocks project documented an environmental approach in the building design and construction of 1.7 million square feet of retail and Class-A office space, residential housing, and parking and is developing an environmental “tenants manual” (already described). Developers and architects for this project report many efficiency features would not have been included if the Alliance had not supported the environmental specialist on the design team. The team regards energy efficiency as a key project achievement and a crucial marketing tool.

Target audiences within the commercial building market consistently report they would like more “hard evidence” about the efficacy of incorporating energy efficiency features into buildings. The Green Buildings and the Brewery Blocks projects are good examples of how such projects can provide hard evidence and be used as case studies for others to emulate. The Alliance can get more “bang for the buck” from these types of projects by promoting their results and benefits. Evaluating the energy savings from these efforts would be especially useful.

Executive Summary


2. Flexible grant programs like State Special Projects and Seed Funding can assist many different activities proposed by an array of entitles that may not fit into other larger, more focused efforts (e.g., Code Support, RPIP).

These smaller efforts in research, building technology, and code development are crucial to “push the envelope” and offer specific, measurable case studies and solutions. Taken together, they can be used to promote the broader objective of market transformation.

WHERE SHOULD THE COMMERCIAL BUILDING INITIATIVE GO FROM HERE?

As illustrated by the conclusions and lessons listed above, the results of EBPI are mixed. In this section we address a broader question:

Ø What market theories and intervention strategies provide the best way to achieve market transformation in the commercial building market?

We think it is important to explore EBPI’s theories because weaknesses in the theoretical underpinnings of EBPI affected both its effectiveness and our evaluation efforts. When talking about theory, we mean concrete descriptions of what happens, and why, in the real world we are trying to influence.

A market theory tries to describe how market forces and actors interact to deal with a product or service – in this case, commercial building construction and renovations. EBPI’s goal is market transformation. Its market theory was that building users are a mass market that can drive demand for better buildings, and if this demand were created, supply would follow the demand, and minimize the importance of “supply-side” actors. Finally, it posited that code activities would support market transformation.

A program theory describes a set of interventions designed to influence the market in a particular direction – in this case, toward greater energy efficiency. EBPI’s program theory proposed the RPIP as the major intervention to activate demand. The RPIP would find the right consumer “hook” (productivity), present it in a mass media campaign, and provide supporting information (e.g., Design Guidelines) and services (e.g., the Advisor Service). The RPIP would create a brand – betterbricks.com – and provide branded information and services, particularly through its website. Belatedly, the RPIP was to facilitate the “handshake” between demand and supply through advocacy and information to supply-side actors. Code

Executive Summary


activities would improve codes and be ready to supply better buildings, but there was little connection between the two major program thrusts.

As is evident in these descriptions, the two major EBPI program theories were disparate and not well integrated. This made it difficult to articulate an overall program logic and to manage and connect program pieces.

Toward a New Theory

EBPI wanted to produce lasting change in the commercial building marketplace. We use three criteria (in question form) developed by Kunkle and Lutzenhiser (2001) to assess whether and why EBPI efforts produced evidence of progress toward lasting market change.

Did EBPI Help Make Energy Efficiency More Relevant and Valuable in the Marketplace?

We do not believe that EBPI’s public information program made energy efficiency more relevant in the marketplace, although that was certainly its intent. The RPIP did not conceive of serving a dynamic, complex market, driven by the view that “buildings are investments” (as described by Kunkle and Lutzenhiser, 2001).

Rather, RPIP focused on the mass market of building users. The RPIP strategy largely left out other powerful actors in the building industry, such developers, real estate professionals, designers, builders, communities, and regulators. It also believed it could provide a “silver bullet” of motivation through mass advertising and a limited amount of in-depth information and hands-on assistance.

Raising the awareness of the relevance of energy efficiency among building users, in the mass market sense, proved to be a difficult and expensive ambition, and not likely to have much impact, since research shows that the majority of building users (i.e., general employees) have little ability to influence space changes. In addition, relying on only one advertising hook – productivity – and only indirectly linking it to energy efficiency limited market relevance for users.

On the code support and advancement side of EBPI, the relevance of energy efficiency did increase for specific communities and actors, as illustrated by the Green Buildings project for the City of Portland, the Brewery Blocks project, and in the code activities in Idaho.

Executive Summary


Did EBPI Help Increase Institutionalized Demand for Energy Efficiency Among Specific Market Actors?

Without increased relevance for energy efficiency, demand was unlikely to increase in the market. RPIP’s marketing treated building users as a mass audience. It made few direct efforts to reach specific types of users who might be more receptive – for instance, government users, owner-users, and build-to-suit users. The shotgun RPIP strategy was not likely to result in what Kunkle and Lutzenhiser call “concrete expressions of willingness to act in particular ways by concrete actors on the ground.”

Where energy efficiency was made relevant to market actors (e.g., to the City of Portland through the Green Buildings project or to developers, designers, and users, as in the Brewery Blocks project), demand has a much better chance of becoming institutionalized.

Did EBPI Help Incorporate the Supply of Energy Efficiency into the Standard Routines of the Building Industry?

The RPIP did not make inroads within the design/development process. The Design Guidelines intend to illustrate how energy efficiency approaches might be made more routine, but these efforts are incomplete. The Advisor Service also may offer a mechanism to take advantage of existing interests in the marketplace, but this service has not gotten much use so far.

On the codes side of EBPI, specific projects may help make efficiency choices more routine. The demand controlled ventilation (DCV) technology development project, hopes to make DCV a standard industry practice. The project manager believes the involvement of a national technical advisory committee is crucial in making DCV routinely chosen. The Green Buildings project will make efficiency choices more routine for Portland city buildings.

Theory Recommendations

We believe the three questions listed above should be applied to any new market and program theories that the Alliance develops for transforming the commercial buildings marketplace. The market theory for change needs to show how it will foster energy efficiency being made more relevant, more institutionalized in terms of demand, and more routine in terms of supply.

Executive Summary


The program theory needs to show how and why its strategies will work in support of changing the market toward greater energy efficiency (and any other goals). This analysis needs to describe: the specific market actors to be targeted; the program mechanisms; the reasons why actors would be interested in change, in general, and using the proposed mechanism, in particular; the best ways to reach market actors; the intersection of market actors with business types; and indicators of how change will measured in the market.

Both Segments and Intersections Matter

In planning strategies for the commercial buildings and renovation market, we believe:

Ø Several useful approaches exist for segmenting the market. Segmentation approaches, however, need to be well defined (e.g., market actors, business types, areas of market opportunity, methods of influence).

Ø It is necessary to see the intersections among market segmentation approaches, so that strategies are both equipped to deal with a complex market and so that they are well integrated.

Ø A system that considers the intersection of market segments – as well as other factors important to the market and funding agency – would help determine the best program approaches.

Segmenting the Market

One way to organize the commercial buildings market is around business or building types – for instance, small retail, groceries, schools, office, etc. This perspective is important to gauge the amount of activity and opportunity in the market and to reinforce the idea that market actors identify themselves with particular industries.

Kunkle and Lutzenhiser’s recent draft report for the Alliance (2001) presents a segmentation approach table taken from the perspective of “areas of opportunity,” or trends relevant to the building industry (e.g., the movement toward more green and sustainable buildings), and then shows how the opportunities intersect with key market actors, existing activities, and mechanisms to reach or affect the market (e.g., marketing, guidelines).

Executive Summary


From a market-change perspective, segmentation by market actors is a key activity since people make decisions, take actions, and organize themselves into groups. It is also people who need to be reached, appealed to, and served. EBPI and other studies have gathered a good deal of information on these actors.

Recommendations – Segmentation and Integration

We would like to see more work on both market segmentation and integration in the commercial buildings market, especially for market actors. We would like to see:

Ø Market actors segmented into smaller, more precise groups that are aligned through common interests and outreach mechanisms, rather than under broad “demand-side” and “supply-side” umbrellas that often prove unwieldy. (For instance, architects are suppliers when a client purchases their services, but almost never supply buildings to the general marketplace. Developers supply buildings to the market, but they demand design/build services from architects and builders.)

Ø Kunkle and Lutzenhiser’s (2001) list of market actors (see below) used as a starting point for finer segmentation, but refined by checking with market experts and by asking: Do actors see themselves as aligned with the roles, as listed, or with some other attribute (e.g., professional kinship or business industry). Their market actor categories are:

• Providers of Capital – Investors (Financial Institutions, Institutional/Pension Funds, Financial Markets, Wall Street), Owner-Occupied, Public Owners, Private Owners

• Developers – Build-to-Sell, Build-to-Hold, Build-to-Suit

• Design and Delivery – Design/Bid/Build, Design/Build, Design-Assist/Construction Manager, Hybrids

• Community/Political/Regulatory Interests – Pro-Development, Progressive, Restrictive

• Real Estate Services Providers – Marketing/Sales, Leasing, Investing, Management/Operations

• Users of Buildings – Lease, Owner Occupied

Executive Summary


Ø Development of consistent terms, definitions, and/or descriptions of these market groups and their roles. If overlap and inconsistencies exist, they should be acknowledged. The lack of agreement about these terms creates considerable confusion.

Second, we would like to integrate the market actor categories with other types of intelligence. We suggest:

Ø Doing a “first cut” to determine the best market actor segments to pursue and how to pursue them, as illustrated in Table ES-1, below. (Note: the contents of this table are for illustration purposes only and should not be relied on for program planning.)

Ø Doing a fuller analysis of “best bet” market actors, expanding on market intelligence and program strategies, and, again, checking with market experts.

WHAT MARKET CHANGE INDICATORS SHOULD WE LOOK FOR?

Market change indicators can only be determined once the market and program theories are fleshed out. However, indicators would need to be consistent with the overall market change criteria suggested by Kunkle and Lutzenhiser and as described above: increased relevance of, increased demand for, and increased supply of energy efficiency in the commercial buildings marketplace. These researchers also suggest looking at energy efficiency indicators to see if more energy efficiency has been built into new buildings or retrofitted into existing ones. Follow-ups to the commercial buildings baseline survey undertaken through EBPI are a logical mechanism for tracking progress in energy efficiency.

Executive Summary


Figure ES-1: Market Actor Analysis (for illustration only)

PROVIDERS OF CAPITAL*

INVESTORS OWNER OCCUPIED

MARKET INTELLIGENCE

FINANCIAL

INSTITUTIONS INSTITUTIONS/

PENSION FUNDS FINANCIAL

MARKETS/ WALL

STREET

REAL ESTATE

INVESTMENT

TRUSTS

PUBLIC OWNERS PRIVATE OWNERS

Key Market Viewpoints

Buildings are investments

Low interest in efficiency


Government sets example Efficiency/ greenness per

se

Buildings are investments Desire for productive

workforce

Good corporate citizens

Business Types ? ? ? ? Offices, schools, hospitals (local, state, federal

government?)

Offices (local, regional, national, international

firms)

Opportunities for Influence

Volatile energy prices

Better income stream, return

Green Buildings


Green buildings


Work environment

Likelihood of Successful Intervention

Low? Low? Low? Low? High Medium

Level of Influence High, initial decisions made here, big picture High influence investors High to medium, depending on visibility of

owner

Continued

Executive Summary




MARKET INTELLIGENCE

FINANCIAL



MARKETS/ WALL

STREET

REAL ESTATE

INVESTMENT

TRUSTS


How to Reach Professional groups, industry pubs, person to person, industry leaders, standards/regulation

Professional groups, industry pubs, peer

influence, press stories, community interests,

public policy, regulation

Professional groups, industry pubs, press stories,

public recognition, regulation

Current Activities ? ? ENERGY STAR Buildings?

ENERGY STAR Buildings?

Green Buildings/ LEED/ Earth Advantage

Brewery Blocks

Other Ideas ? Socially Responsible Investing?

Socially Responsible Investing?


? ?

Champions, Leads, Projects

? ? ENERGY STAR

Buildings investment

tool?

ENERGY STAR Building

Partners?

LEED Buildings, Requirements to build to

LEED standards?

Owners that are using LEED?

Need for Primary Research

Maybe Yes Yes Yes Probably not Yes, focus on large institutional users

Pursue Further Maybe research

Yes, research Yes, research Yes, research Yes, high priority Yes, high priority

* Question marks signal lack of knowledge or barriers

Executive Summary


EBPI – MPER #1 DETHMAN & TANGORA h 1

CHAPTER 1: THE EFFICIENT BUILDING PRACTICES INITIATIVE

INTRODUCTION

The Northwest Energy Efficiency Alliance (Alliance) is a non-profit group of electric utilities, state governments, public interest groups and industry representatives committed to bringing affordable, energy-efficient products and services to the marketplace. The Alliance’s Efficient Building Practices Initiative (EBPI) couples a Regional Public Information Program (RPIP) with several building code support programs to:

Ø Increase consumer demand for highly energy efficient commercial buildings.

Ø Enhance the capability of code-related institutions so they can meet increased consumer demand for efficient buildings.

The Alliance hopes this pairing of consumer demand and codes infrastructure enhancements will ensure greater compliance with current energy codes and propel voluntary “standard practice” beyond them. The remainder of this chapter presents a history of EBPI, explains its elements, and summarizes the findings from the Final Baseline Report – Efficient Building Practices Initiative (March 2001).

HISTORY OF EBPI

In this section we present a short history of the Northwest Energy Efficiency Alliance’s Efficient Building Practices Initiative. In 1997, the Alliance Board commissioned in-depth research to “identify strategies that the Alliance could pursue which would result in sustainable and effective energy codes” (Heschong Mahone Group, Report #E98-009, April 1998). This study was integral to the Alliance’s view that energy codes are a crucial ingredient in its “portfolio” of investments in energy efficiency and market transformation in the Northwest.

Energy codes, while different in each state, are generally strong in the region. Codes also provide a mechanism for facilitating widespread adoption of new practices as technologies and practices improve. However, as a Board member associated with the evolution of EBPI pointed out, “Codes were drums whose music was not appreciated by every ear in the region;” and in 1997, as EBPI was first

Chapter 1: The Efficient Building Practices Initiative

DETHMAN & TANGORA h EBPI – MPER #1 2

being envisioned, there were two reasons to downplay what became known as the “C” word:

1. In Oregon and Washington, some thought incremental efficiency improvements above the ambitious codes that already existed would involve changes in design, behavior, or other aspects of construction that could not be codified.

2. In Idaho and Montana, building industry and political powers tended to oppose any coercive codes urged by people outside their states.

The Heschong Mahone research was focused on how to sustain effective energy codes, yet it also had to temper a code focus within the context of these two political and market realities. Thus, the Alliance Board appointed some of its members to serve on a Board Management Committee to oversee the research. This committee interacted extensively with the researchers, discussing the results from, among other tasks, interviews with over 100 energy stakeholders in the region and a daylong meeting with about 50 others.

The committee accepted most of the suggestions stemming from the Heschong Mahone research and formulated the multi-part EBPI project, which they presented and defended to the rest of the Board. EBPI called for seven interrelated elements, implemented over three years, to support energy efficiency and codes in the Northwest. The Board approved these elements in February 1998, and directed the committee (renamed the EBPI Steering Committee) to oversee implementation of EBPI, giving them considerable autonomy to make decisions and guide the program. The current names for the seven EBPI program elements are shown in Table 1. Not all elements are presently in operation.

Table 1: Current Names of EBPI Elements

EBPI PROGRAM ELEMENTS

1. Regional Public Information Program 2. New Construction Baseline 3. Energy Code Support 4. State Special Projects 5. Seed Funding 6. Transition Funding for Infrastructure 7. Code Advisory Committee



EBPI PROGRAM ELEMENTS

In this section we briefly describe each EBPI element.

Regional Public Information Program (RPIP)

The Regional Public Information Program (RPIP) has always been the largest and most visible element within EBPI, commanding more than half of its $6 million resources. Heschong Mahone’s original project plan sought to “create a strong market pull...based on demand for the benefits conferred by energy codes” (Heschong, p.40). It recommended that a “savvy public relations firm” be hired to promote code benefits with the general public, new homebuyers, commercial owners and tenants, and government officials who manage building departments.

RPIP’s core strategies1are to:

Ø Identify the benefits of implementing energy efficient building practices that have sufficient value to motivate the target audiences.

Ø Identify and address barriers to awareness and action by target audiences.

Ø Address both benefits and barriers through a coordinated marketing and information program.

Ø Overcome the inertia that is presently inhibiting the marketplace from achieving its potential to implement energy efficient building practices.

Ø Increase awareness and acceptance of the end-user benefits of energy efficient building practices. Leverage this awareness and acceptance into actions that result in demand for and implementation of more energy efficient building practices in the marketplace.

Ø Address both the push and pull sides of the marketing equation – end-users and the audiences from whom they seek energy efficient practices.

These core strategies led to a competitive solicitation process that resulted in the Alliance contracting with Cole & Weber, a Northwest-based advertising firm, to develop a regional public information campaign. The EBPI Steering Committee

1 These strategies are located in the project description on the Alliance website

http://nwalliance.org/coordination/



asked Cole & Weber to conduct market research to help develop a message for the campaign. The market research was done in the spring of 1999, and consisted of in-depth interviews, focus groups, and a review of the literature. Interview and focus group participants included supply-side service providers such as architects, engineers, and real estate professionals, and demand-side purchasers such as homeowners, business owners, and influential commercial business employees.

The market research revealed that energy efficiency was not “top-of-mind” with any of the groups included in the research. Furthermore, the residential groups did not strongly resonate with potential motivators and benefits of energy efficiency, such as sustainability or an improved environment. The supply-side focus groups revealed they were somewhat skeptical of being able to sell energy efficiency in buildings. Only with business respondents did the research reveal a “powerful hook" – connecting energy efficient buildings with increased worker productivity.

Figure 1 lists the lessons of the initial market research that Cole & Weber presented to the EBPI Steering Committee (October 28, 1999, slide presentation):

Figure 1: Lessons Learned from Initial RPIP Research

During this presentation, Cole &Weber also presented their recommended strategies for RPIP:

1. Focus upon the commercial sector only.

2. Target primarily employees and decision-makers in businesses, with a secondary focus on architects, developers, real estate brokers, and government agencies.

What have we learned?

Ø It’s “me” not “we”

Ø Productivity has power

Ø Residential: fewer opportunities

Ø Beware of the supply side



3. Develop a brand focus, taking a business point of view, centered on productivity and empowerment – “Evangelists for a better way to work.” Energy efficiency, while still the concern, would not be in the central campaign message.

4. Develop a product – a website that would “give them a place to go.” This product would be bolstered by more traditional public information efforts to “drive” target audiences to go to the website.

Campaign Assumptions

The EBPI Steering Committee agreed to this strategy and Cole & Weber proceeded to develop a business model for the public information campaign, along with a statement of objectives for each of the components (see Figure 2).

Figure 2: RPIP Business Model

The business model is predicated on building “brand awareness” for the website betterbricks.com, which then leads the end-user to request help with creating highly



energy efficient workspaces or buildings using Design Guidelines available through betterbricks.com.2

The components of the campaign (primarily directed to the demand-side audiences) included mass media and print advertising, a public relations effort, the betterbricks.com website (including a Help Desk with on-site consulting available through an advisory service), material for the website, and distribution of informational materials (such as brochures) to interested decision-makers. An Advocacy Program was targeted to supply-side actors. The following describes Cole & Weber’s objectives for each of these components.3

Ø What the advertising does:

• Builds awareness of the “brand” (betterbricks.com).

• Builds awareness of the message that “space affects productivity.”

− Challenge perceptions of what affects productivity

− The building is a more fundamental and important consideration than many of the trendy management measures in which companies invest their resources

• Drives people to the website.

Ø What the PR does:

• Builds awareness of the brand.

• Builds awareness of the message that space affects productivity.

• Provides credibility for ad claims.

• Drives people to the site.

Ø What the website does:

• Moves audience from an initial awareness of the “space matters” message to a deeper sense of understanding and validation.

2 December 14, 1999, Cole and Weber presentation to the Alliance Board of Directors.

3 February 18, 2000, Cole and Weber presentation to the EBPI Steering Committee.



• Addresses the needs of those who wish to know more about and pursue high efficiency buildings and workspaces (the “purchase path”).

• Offers audiences an avenue to more personal involvement with the brand through the Help Desk and Advisors.

Ø What brochures and other supporting information (collateral) do:

• Provide a concise (“Reader’s Digest”) version of key web components.

• Make the brand more real, tangible.

• Help “incubate” a high efficiency building or workspace.

• Equip Help Desk.

Ø What the advocacy program does:

• Spreads the word to suppliers across the region

− Connection between space and productivity

− Connection between EE practices and good space

− Demand is coming, there’s business to gain

− Suppliers can turn to BB.com for “how to” help in responding to that demand

• Increases likelihood that suppliers contacted about BB.com “cold” will respond favorably

• Distributes Design Guidelines

• Broadens the circle of “advocates” beyond the small club that now exists

A review of the key assumptions laid out by Cole & Weber makes it clear that by the May 1, 2000, launch of the betterbricks.com campaign, RPIP had evolved to focus on the betterbricks.com brand and its associated elements – a referral service and Design Guidelines (still in development). While the demand-side components of the business model were planned and, for the most part, launched, the Advocacy



Program for the supply-side – which was to include ads, mailers, a speaker series, and a toolkit – was delayed and never implemented. This was due in part to the unfinished Design Guidelines that were also were an integral part of the supply-side efforts.

New Construction Baseline

The goals of the New Construction Baseline survey were to:

Ø Document new construction current practices in the residential and nonresidential sectors in all four Northwest states, and the reasons behind these practices.

Ø Establish a measurement protocol that is repeatable in five years to measure market progress.

A sub-goal of the New Construction Baseline was to test the degree to which codes were effectively influencing construction practices across the region.

The methods for this Baseline Survey included random sampling procedures to ensure high reliability of results, review of permit covers, plan checks, and site visits to commercial and residential locations under construction. General contractors, owners, architects, and engineers from selected buildings were also interviewed.

All design and data gathering for this element of EBPI were completed in late winter 2000. The project resulted in the development of baselines for:

Ø Residential single-family buildings across all four states

Ø Multi-family buildings in Oregon and Washington

Ø Nonresidential buildings – for all four states, a sample frame of buildings was identified and interviews were conducted with key actors to gather the reasons behind current building practices; however buildings were not reviewed in Washington because a baseline study was completed there in 1996

The reports were finalized in spring 2001 and short summaries are being prepared.



Energy Code Support

This element of EBPI provides on-going support for code maintenance and development, helps ensure that a code infrastructure and services will continue to exist, and keeps institutional memory intact within the region. It also gives the Alliance information about code maintenance and developments, as well as ongoing access to the people in charge. Energy code support funding has been provided to the appropriate agencies in the three states with code support functions: Oregon, Washington, and Idaho. To date, Montana has not asked for, nor received, energy code support funding from the Alliance.

State Special Projects Program

This element of EBPI reviews proposals and grants funding for special energy code and building energy efficiency projects from state energy and code agencies, code organizations, non-profit organizations, public agencies and private industry. The Alliance’s overall goals for these projects are to improve the process of code compliance, encourage voluntary actions that go beyond code, and foster partnerships in the building industry. A variety of State Special Projects are planned or underway, including:

Ø The Oregon Office of Energy is researching the efficacy of using CO2

Sensor-Based Demand Controlled Ventilation in assembly spaces and, if feasible, developing code language specifying its use.

Ø The Idaho Department of Water Resources is conducting a project designed to demonstrate that residential builders and consumers would accept the Home Energy Rating System.

Ø The Idaho Chapter of the International Conference of Building Officials is partnering with cities and counties in southern Idaho to help them adopt local codes, set up building departments, and train staff.

Ø Shorebank Pacific, using real projects in Portland, complemented its loans on low-income housing rehabs by training Portland Development Commission staff in energy efficiency and life-cycle costing.

Ø The City of Portland Energy Office developed four demonstrations and created persuasive information to support life-cycle cost analysis in publicly-funded or assisted buildings.



Ø The Oregon Building Officials Association is placing Oregon energy code interpretations and code compliance documents on a website.

Ø The Oregon Office of Energy developed prescriptive duct sealing code language for residential buildings that was submitted to the State Codes Advisory Board, but not adopted due to building industry opposition. Efforts are being made to seek support and resubmit the language.

Ø A Special Plans Examiner/Inspector Certification project helped maintain the private sector program in Washington. The funding helped support a round of exams, a database for referrals, and transfer of the program to a Washington code agency.

Seed Funding

This aspect of EBPI was designed to provide seed funding for businesses that intend to be self-sustaining and that deliver services to assist in implementing energy codes at the design and build levels. Support of these businesses is intended to help improve energy code compliance and upgrade building energy efficiency beyond minimum code requirements.

Over time, much of the budget allocated to Seed Funding has been transferred to other components of EBPI (particularly RPIP), or has been allocated to a sub-category termed “Other Projects” under the umbrella of Seed Funding. The “Other Projects” category includes: membership in the New Buildings Institute; sponsorship of the Future @ Work project; funding for various aspects of the Brewery Blocks projects; and support for a few small consultant contracts. While these projects may not meet, at least at this point, the intention to be self-sustaining, they are private-sector efforts aimed at enhancing energy efficiency.

Transition Funding for Infrastructure

Early on in EBPI, short-term bridge funding was provided to support energy code efforts in appropriate agencies in Montana, Oregon, Idaho, and Washington when the Bonneville Power Administration and utilities scaled back their support of these agencies. Further funding for ongoing support of code efforts was then continued under the Energy Code Support element, described above.



Code Advisory Committee

The Code Advisory Committee was to be a region-wide body of code stakeholders that would advise the Alliance and staff on EBPI code-related decisions. It was never implemented because:

Ø U.S. DOE set up a program to bring together code staff on a quarterly basis (the Regional Code Discussion Group); and

Ø EBPI’s lack of direct emphasis on codes decreased the usefulness of an advisory committee focused on codes.

KEY FINDINGS FROM THE BASELINE REPORT

The Final Baseline Report—EBPI presented key findings from surveys of demand-side actors (commercial workers) and supply-side actors (architects and real estate developers). We completed these baseline surveys during April and May 2000. Demand-side surveys were completed by the first day of the launch of the betterbricks.com advertising campaign; the supply-side interviews were completed during the first two weeks of the campaign.

Commercial Employee Interviews

To establish the baseline for commercial employees, we surveyed random samples of three primary target audiences of the betterbricks.com public information campaign: general employees, influential staff who shape workspace decisions, and decision-makers who determine final outcomes. Each sample had about 500 respondents. The surveys investigated key company characteristics, opinions about the effects of the physical workspace on productivity, actions taken to improve productivity, sources of information about workspace and productivity, and Internet use and TV viewing patterns, including whether they had seen any advertising or news stories relating the workspace to productivity.

Key baseline market viewpoints, some of which are compared with follow-up data in Chapter 3, included:

Ø Influential staff and decision-makers were more positive than general employees about their companies “being concerned about worker productivity.”



Ø About 60% of all respondents believed the physical workspace strongly affected productivity.

Ø When asked what physical factors have a large effect on employee productivity, about 45% in each group said the “layout of the workspace,” with “equipment” a distant second (about 22%).

Ø Between 10% and 20% of respondents spontaneously mentioned energy-related workspace design features – natural light, good lighting, good temperature – as strong influences on productivity.

Ø When spontaneous mentions and prompted ratings of features relevant to energy efficiency were combined, 90% of respondents thought lighting strongly affected productivity, 75% thought temperature did, and 55% thought natural light did.

Ø Decision-makers making workspace changes rated “increased productivity” as their top priority among six potential reasons to change the workspace (79% gave “important” or “very important” ratings).

Ø Decision-makers said they would use external sources to gather information about workspace decisions and productivity. In contrast, employees most often mentioned they would seek information from within the company. Influential staff fell mid-way between the two groups.

Ø About 20% in each of the three groups would turn to the Internet for more information on the relationship between the workspace and employee productivity.

Ø About 70% of general employees, influential staff, and decision-makers report using the Internet on the job, with more than half of these users reporting that they access the Internet for four hours or less per week.

Ø Over 95% of respondents typically watch some TV during the week. About 90% of respondents watch the news during the week, about 80% watch prime time TV, and fewer than 60% watch sports.

Ø Data show that few respondents, if any, had seen the betterbricks.com advertising.



Architect and Developer Interviews

To gather baseline data on these groups, we interviewed over 100 commercial workspace architects and developers in the Pacific Northwest. Architects and developers of commercial space are key players in generating, understanding and meeting the demand for buildings with high energy-efficient features. These groups were secondary target audiences of the betterbricks.com efforts. The interviews explored the roles these actors play in commercial workspace design, their experience in designing buildings to enhance worker productivity, the extent to which energy efficiency elements are included to enhance productivity, and the sources they use if they need information about workspace and productivity.

Key findings from these interviews included:

Ø Both architects and developers were equally concerned with the building envelope and the general mechanical systems of interest to EBPI (e.g., natural light and temperature control), and with lowering operating costs.

Ø Architects are more likely than developers to be concerned about the “look” and aesthetic appeal of the buildings and workspace, meeting energy efficiency and environmental goals, dealing with interior lighting design and layout, and addressing the “human” factors of occupying a building (e.g., physical comfort, productivity, worker satisfaction).

Ø Seventy percent of architects spontaneously mentioned good lighting when asked to name the features of the physical workspace that contribute most to employee satisfaction and productivity. About 60% mentioned natural light and about 50% mentioned a comfortable temperature. Developers mentioned these features less often, with about 40% of developers naming natural light and comfortable temperature and about 30% naming good lighting.

Ø Architects, more than developers, were able to name specific natural lighting and artificial lighting strategies that enhance employee satisfaction and productivity, but both groups were equally able to name heating and cooling equipment that would improve productivity and said that good temperature control was one of the hardest objectives to achieve in commercial buildings.

Ø Architects are most likely to turn to reference materials, including the Internet, to learn more about the relationship between workspace design and worker satisfaction and productivity, while developers are most likely to turn to architects and other design and construction professionals.



However, many developers suggested that it was unlikely that they would pursue such information.

Ø Almost one-third of architects and developers said they had seen advertising or news stories about improving worker satisfaction and productivity through workspace design. None, however, were able to specifically identify the just recently launched betterbricks.com advertisements.

Conclusions and Issues

Data from the baseline surveys revealed some important challenges betterbricks.com was likely to face in reaching its primary and secondary target audiences, including:

1. The betterbricks.com message is competing with other messages and associations with productivity that are stronger than those related to energy efficiency, such as the layout of the workspace and office equipment. Linking productivity to energy efficiency actions and making that linkage more visible will be key to the success of the betterbricks.com campaign.

2. Betterbricks.com messages may not currently attract developers since the latter do not tend to be concerned with interior workspace design, an emphasis of the campaign. Betterbricks.com will need to hook developers with issues that concern them, such as providing reliably comfortable temperatures.

Preferences for information sources appear to vary by target audience, so that broad media approaches (such as television and print ads in general audience papers and magazines) may be less effective than more targeted campaigns. If broad media approaches are used, they should be carefully selected to reach the specific target audiences.


CHAPTER 2: THE EBPI EVALUATION APPROACH

In developing an evaluation approach for EBPI, we wanted to provide insights into how well EBPI’s dual demand and code-related strategies were working. The logic model, discussed below, attempts to explain the underlying interconnections and desired short- and long-term outcomes of the program. The second section of this chapter charts the EBPI evaluation components and timeline to date. Finally, we briefly discuss some of the challenges of trying to evaluate, in the short-term, a project that hopes to affect building practices over the next ten to twenty years.

THE EBPI LOGIC MODEL

Figure 3, on the next page, shows a “logic model” that the Steering Committee developed for EBPI. This model shows the intended links between the program elements and the desired short-term program outcomes through the end of the current contract period (mid-2001), as well as the desired market transformation results in the long-term (20 years). It is a graphical representation of the thinking behind EBPI and also frames the EBPI evaluation efforts. Note, however, that one EBPI element – the Code Advisory Committee – has not been implemented. This element is connected to every other program element and to several short-term outcomes. In Chapter 11, Conclusions and Lessons for EBPI, we briefly discuss how this missing element affected the program.

As shown in the logic model, the elements of the program are interconnected (or were intended to be interconnected) in a variety of ways, so that the results of various efforts inform and support one another. Of greatest interest to the current evaluation efforts are the desired “Contract Period Outcomes,” since meeting, or not meeting, these outcomes defines the performance of EBPI in the short-term and helps predict long-term results as well. These short-term outcomes will be used as our progress indicators in Chapter 10.

Chapter 2: The EBPI Evaluation Approach


Figure 3: Logic Model for EBPI

EFFICIENT BUILDING PRACTICES INIT IATIVELinks Between Elements, Contract Per iod Outcomes, and MT Outcomes Logic Model

KEY:= pr imary= direct= secondary= indirect

* = may be expanded

Long-TermOutcomes

All NewConstruction AndMajor RenovationResidential And

CommercialBuildings Meet aMinimum Level ofEnergy Efficiency

a n d M o s tBuildings GoBeyond That

Minimum Level

Contract Period Outcomes(by mid 2001)

Incrementally improved market place(=demand) for professionals with energy

efficiency expertise exists

More Efficient andAppealing

Buildings ExistBecause

Occupants andOwners Want

Them

Public (primarily non-residential)awareness/ curiosity of productivity

opportunit ies in buildings and how theyrelate to energy efficiency is raised

Increased availabil ity of energy efficiencyprofessionals for folks to call

Decreased dependence on outside fundingto support ee activities in buildings =

moving toward self-sufficiency

As a response to consumer requests,supply-side professionals touched by thisprogram offer energy eff iciency solutions

Energy eff iciency professionals’ responseto consumer request is increased energy

efficiency

Maintain current code (=sti l l viable)

The EBPI is seen as a regional effort

A strategy and reason exists to continuethe project

RPIP

Baseline

EnergyCode

Support

State *Specia l

Projects

Seed*Funding

AdvisoryGroup*

Elements


EFFICIENT BUILDING PRACTICES INITIATIVE DETHMAN & TANGORA h 17

EVALUATION COMPONENTS FOR EBPI

Table 2 shows key elements of the EBPI schedule and the components and schedule of the EBPI evaluation activities.

CHALLENGES TO EVALUATING EBPI

Measuring the progress of EBPI presents challenges. As the Logic Model (Figure 3) illustrates, EBPI is a large, multi-faceted program with many interconnections that were designed to be implemented over at least ten years. In addition, the listed short and long-term outcomes are ambitious and fairly general. Finally, while we are technically in the third year of EBPI, most program elements have been operating for a little over a year-and-a-half, and the largest program element, the RPIP, was not launched until May 2000, less than a year ago. Moreover, due to concerns about its effectiveness, the television-advertising component of the RPIP was reduced from three to two waves.

Our evaluation approach has tried to measure the performance of EBPI components in its start-up period, and, at the same time, provide progress indicators that can be clearly linked to long-term impacts. We have provided feedback on how specific aspects of EBPI are working (e.g., the betterbricks.com usability testing and the advertising focus groups) and we have also tried to glimpse how the program is likely to progress over time (e.g., baseline and follow-up surveys and process evaluations of codes support).

However, we would like the readers to bear in mind that changing demand and supply in the commercial building market, and changing energy codes, tends to be incremental, does not progress in equally-sized steps, and may be affected by significant outside forces (e.g., economic changes or an energy crisis). Thus, determining whether or not progress has been made may not, in some cases, be knowable or definitive. Nonetheless we feel this evaluation has revealed a number of important lessons that we will discuss in the Conclusions chapter of this report.



Table 2: Key EBPI Events and Evaluation Components and Schedule

EBPI EVALUATION

1. End of1997 – Heschong Mahone Study

2. Early 1998 – Board Approval of EBPI Elements; Hiring Project Manager; Hiring of Cole & Weber

3. Mid1998 to Mid-1999 – EBPI planning and RFPs for non-RPIP elements; Cole & Weber Background Research and RPIP Planning

4. May 2000 – Launch of betterbricks.com campaign, including advertising, marketing, public relations, website

5. May to July 2000 – Reporting of website “bugs”

6. May to September 2000 – Betterbricks.com TV ad waves and other campaign activities (e.g., print, PR)

7. October 2000 – November wave of TV ads cancelled

8. October to December 2000 – Revamping of website

9. January to June 2001 – New strategy development for EBPI

Ø EBPI Baseline Report (draft August 2000; final March 2001) included:

• Program and evaluation component overview

• Program history • Results from surveys conducted to

measure baseline for betterbricks.com campaign:

− General commercial employees − Influential staff in commercial

businesses − Business decision-makers − Building supply-side actors –

architects and developers

Ø Draft Internal White Paper Report (November 2000) included:

• Recap of Baseline MPER key insights • Results of usability tests of the

betterbricks.com website (September 2000)

• Results of focus groups on betterbricks.com advertising (October 2000)

• Review of betterbricks.com website use (October 2000)

• Conclusions and recommendations to date

Ø Draft EBPI Market Progress Evaluation Report #1 (April 2001) included:

• Recap of baseline evaluation results • Results of follow -up surveys of

betterbricks.com target audiences (November 2000; March 2001)

• Process evaluations of Codes Support, State Special Projects, and Seed Funding projects (March 2001)

• Performance Indicators • Conclusions and recommendations to

date


CHAPTER 3: DEMAND-SIDE RESPONSE TO THE RPIP

BACKGROUND AND PURPOSE

The RPIP campaign targeted commercial employees, decision-makers, and influential staff with television and print advertisements. We conducted a follow-up survey with decision-makers and influential staff in October 2000, at the end of the 2000 television ad campaign (approximately six months after the campaign began). The results of this follow-up survey were compared with the baseline survey conducted in April 2000.

The follow-up survey sought to assess respondents’ awareness of the RPIP betterbricks.com branding campaign. After asking questions to ensure that we had reached a population comparable to that utilized for the baseline study, we explored the following areas:

Ø Awareness of, and response to, advertising or news stories about a website that provides information on improving productivity in commercial buildings;

Ø Opinions about the effect that the physical workspace has on employee productivity;

Ø Information sources respondents would use to learn more about how to improve productivity in their workspace; and

Ø Usage patterns of publications and the Internet.

METHODOLOGY

The methodology for the follow-up survey includes the sampling plan and the development of questions and methodology for assessing awareness.

Sampling Plan

We generated the sample for the follow-up survey in the same manner as for the baseline survey. We purchased a list of commercial establishments located in the four-state Pacific Northwest and selected SIC codes corresponding with offices and

Chapter 3: Demand-Side Response to the RPIP


schools, wholesale and retail stores, health care establishments, and financial institutions. We identified those with more than 20 employees and called the selected businesses. Once we reached someone, we asked to speak to “a person in your organization who makes or influences decisions about the space where your employees work.” Once we were connected with the individual, we verified that he or she did make or influence such decisions. This screening process determined whether the respondent was a "decision-maker" or "influential-staff."4

For this follow-up survey, we allocated the sample roughly evenly between newly-contacted respondents and respondents with whom we had spoken in the baseline study. By re-contacting those with whom we had previously spoken, we could employ a panel method to assess changes in attitudes over time. Table 3 shows the sample distribution in terms of decision-makers and influential staff, and new and repeat respondents.

Table 3: Follow-Up Sample Distribution by Respondent Type (n=1,018)

INTERVIEW GROUP POSITION IN FIRM

NEW GROUP REPEAT GROUP

TOTAL

Decision-Maker 321 200 521

Influential Staff 249 248 497

Total 570 448 1,018

Methodology for Assessing Awareness

During the period between the baseline and follow-up studies, advertisements for betterbricks.com appeared both on television and in business and trade publications. The questionnaire posed a series of questions to assess whether our sample recalled seeing any betterbricks.com ads.

4 Though the baseline includes general commercial employees as well as decision-makers and influential staff,

general commercial employees were not included in the follow-up because those who participated in the usability study (see Chapter 6) did not feel that the ads or website were designed for them. This, coupled with indications from the volume of website traffic and help desk use, suggested that resources should be focused on the most important actors targeted by the campaign – the influential staff and decision-makers.



We included a set of questions to obtain unaided recall of the ads and the ad sponsor, betterbricks.com, asking whether the contact had "seen or heard any advertising or news stories about a website that provides information on improving productivity in commercial buildings.” We followed up with those who said they had, asking where they recalled seeing the ad or story, what they recalled seeing, whether they recalled the sponsor, and, if so, who the sponsor was. This set of questions enabled us to assess whether or not the respondents definitely or likely saw a betterbricks.com ad or news story and whether or not they could recall the betterbricks.com name on their own.

We also developed a set of questions to obtain aided recall of the website name for all other respondents. We asked if they had “seen or heard anything about an organization or website called betterbricks.com.” Of respondents who said they had, we asked where they had seen the information and what they recalled seeing. As we did using the unaided question set, we assessed whether or not respondents definitely or likely knew of betterbricks.com.

We determined those who did not recall a betterbricks.com ad or story at the time of the interview by their response to the aided and unaided questions. Either the respondent said they did not recall such an ad or story or said that they did recall such an ad but then could provide no details or provided incorrect details (e.g., “It was telling you about workshops you could attend to learn how to improve employee productivity.”).

SAMPLE CHARACTERISTICS

The sample for the follow-up study is comparable to the baseline sample across all firm characteristics. We found no statistical differences between the two samples for each of the firm characteristics we explored. Further analyses also found no statistical differences between decision-makers and influential staff, so the data reported here do not distinguish between the two. We also found no differences in sample characteristics between the new group and the repeat group of respondents.

The baseline sample includes 1,037 respondents and the follow-up sample includes 1,018 respondents. The follow-up sample includes two subgroups: the new group comprised of 570 respondents who were newly surveyed, and the repeat group comprised of 448 respondents who had also been surveyed for the baseline study (see Table 4).



Table 4: Comparison of Baseline and Follow-up Organizational Characteristics

ORGANIZATIONAL CHARACTERISTIC

BASELINE PERCENT* (N=1,037)

FOLLOW-UP PERCENT

(N =1,018)

NEW GROUP PERCENT**

(N=570)

REPEAT GROUP PERCENT**

(N=448)

STATE

Idaho 9 9 10 8

Montana 17 17 14 20

Oregon 31 32 31 33

Washington 43 42 45 39

SECTOR

Office/School 55 56 56 56

Wholesale/Retail 28 25 25 25

Health Care 9 11 10 12

Financial Institution 8 8 9 7

NUMBER OF EMPLOYEES

20-49 56 53 53 54

50-99 26 27 25 30

100-249 12 12 13 10

250-449 3 4 4 4

500+ 2 4 5 2

AGE OF BUILDING

< 5 Years 8 8 10 6

5-20 Years 30 30 31 28

>20 Years 59 60 56 65

Don’t Know 3 2 3 1

Continued



ORGANIZATIONAL CHARACTERISTIC

BASELINE PERCENT* (N=1,037)

FOLLOW-UP PERCENT

(N =1,018)

NEW GROUP PERCENT**

(N=570)


(N=448)

OWNERSHIP

Own Facility 63 63 63 65

Lease Facility 35 35 35 33

Don’t Know 2 2 2 2

SQUARE FOOTAGE

Under 10,000 20 22 24 19

10,000 – 25,000 22 23 21 26

25,000 to 50,000 16 15 14 16

50,000 – 100,000 8 8 9 6

100,000+ 9 12 13 10

Don’t Know 25 21 19 23

* Results from the baseline include only influential and decision-maker responses

**The new and repeat groups are subgroups of the follow-up survey sample

In the baseline survey we asked decision-makers, but not influential staff, whether their firm was currently in the process of planning to, or making changes to their workspace. We asked both influential staff and decision-makers about this in the follow-up. As with other firm characteristics, we found no statistical difference between the groups (see Table 5).



Table 5: Plans to Change Workspace (Multiple Responses Permitted for Types of Change)

PLAN TO CHANGE WORKSPACE

BASELINE PERCENT* (N=516)

FOLLOW-UP PERCENT

(N =1,018)

NEW GROUP PERCENT**

(N=448)


(N=570)

Renovating Current Space 27 30 31 28

Constructing New Space 11 15 14 15

Moving to New Leased Space 10 8 8 8

No Changes Planned 67 61 61 63

* The baseline survey only asked decision-makers about plans to change workspace.

**The new and repeat groups are subgroups of the Follow-up survey sample.

AWARENESS OF THE BETTERBRICKS.COM AD CAMPAIGN

Awareness of the RPIP campaign is necessary before the campaign can influence behavior. Additionally, a primary outcome from the contract period (see Chapter 10) is to increase the awareness/curiosity of the nonresidential public to "the productivity opportunities in buildings and how they relate to energy efficiency." Therefore, the primary questions for this follow-up were:

1. Whether awareness of the RPIP betterbricks.com campaign had increased above zero since the inception of the project; and

2. Whether awareness of productivity opportunities related to energy efficiency building solutions had increased above the point measured in the baseline survey.

Awareness of the Ad Campaign and of the betterbricks.com Name

As shown in Table 6, the entire follow-up sample of 1,018, using aided and unaided recall, yields a total of 53 (5.2%) who were aware of betterbricks.com through advertisements or news stories. Of these 53, 39 respondents clearly or likely recalled the betterbricks.com ad campaign and recalled the betterbricks.com name in unaided questioning (11 respondents) or aided questioning (28 respondents). Another 14 respondents recalled the ad but did not recognize the betterbricks.com name in aided questioning.



Table 6: Recall of betterbricks.com by Interview Group (n=1,018)

RECALL OF BETTERBRICKS.COM NAME QUALITY OF AD CONTENT RECALL

UNAIDED NAME

RECALL AIDED NAME

RECALL NO NAME

RECALL

TOTAL

ENTIRE FOLLOW-UP SAMPLE (N=1,018)

Total Clearly or Likely Recalling 11 (1.1%) 28 (2.8%) 14 (1.4%) 53 (5.2%)

No Ad Recall NA NA NA 965 (94.8%)

NEW GROUP (N=570)

Clearly or Likely Recalling 2 (0.4%) 10 (1.8%) 4 (0.7%) 16 (2.8%)


REPEAT GROUP (N=448)

Clear or Likely Recalling 9 (2.0%) 18 (4.0%) 10 (2.2%) 37 (8.3%)


Table 6 also details awareness by surveyed group. Notably, 70% (37 of 53) of those who were aware had participated in the baseline survey. This finding indicates that the baseline survey’s focus on productivity in the workspace primed the baseline study participants to subsequently notice information on the subject.

As a consequence of the effect of prior interviewing, awareness for the general population is best represented by awareness for the newly-interviewed group. In the new group, about 3% (16 of 570) could recall a betterbricks.com ad and 2% (12 of 570) could recall both the ad and the sponsor.

Awareness by Advertising Media

Of the 53 respondents identified in Table 6, 49 (92%) respondents reported seeing the ad on television (two of whom also saw print ads). Three people who cited magazines or journals as the sole source of the ad they remembered were among those who did not recognize the betterbricks.com name when aided. One person who provided unaided mention of betterbricks.com and said that he had visited the website no longer remembered what advertisement led him to the site.



Awareness by Respondent Characteristics

Respondents from firms planning to move to new leased space were twice as likely as others to be aware of the ads (10% versus 5%, x2, p = .05).

Respondents planning to make any changes to their space were more likely to be aware of the ads than those who were not (6% versus 4%), however the difference is not statistically significant.

Awareness did not vary significantly by the type of business (e.g., office) in which the respondent worked, or by the number of employees.

Interest Generated by Campaign

We asked the 39 who clearly or likely recalled an ad or news story and also recalled the betterbricks.com name to select a statement that “best fits your interest in betterbricks.com.”5 As shown in Table 7, 10% were very interested in betterbricks.com after seeing an ad, and about 50% were somewhat interested. Differences between the two groups do exist, but they are not statistically significant.

Table 7: Interest Level for Respondents Recalling betterbricks.com (n=39)

INTEREST LEVEL IN BETTERBRICKS.COM NEW GROUP REPEAT GROUP

TOTAL

Very Interested in What They Are Saying 2 (5%) 2 (5%) 4 (10%)

Somewhat Interested in What They Are Saying 4 (10%) 16 (41%) 20 (51%)

Not Much Interested in What They Are Saying 3 (8%) 4 (10%) 7 (18%)

Don’t Recall Enough About What They Are Saying to Have an Opinion

3 (8%) 5 (13%) 8 (21%)

Total 12 (31%) 27 (69%) 39 (100%)

Note: All percentages based on total n=39

5 Only those who recall the betterbricks.com name can pursue any interest the ad might have generated for

them concerning the website.



Twenty-four of the 39 respondents came from either the office or schools category, while 15 came from other businesses. Respondents from office and school environments were more likely to be very or somewhat interested in the ad (18 of 24) than were respondents from other business types (6 of 15, x2, p = .03). Those respondents making changes to their space were only slightly more likely to be interested than those not making changes.

Visits to Website

The 39 respondents who clearly or likely recalled an ad or news story and also recalled the betterbricks.com name were the only respondents who could reasonably be expected to visit the betterbricks.com website. We asked these 39 whether they had visited the site. The responses for all 39 respondents who were aware of the ads are reported in Table 8.

Table 8: Number of Website Visits and Planned Visits by Those Aware of betterbricks.com (n=39)

TYPE OF RECALL OF betterbricks.com

VISITED WEBSITE

PLAN TO VISIT IN 2 WEEKS

PLAN TO VISIT IN 3 MONTHS

NO PLANS OR DON’T KNOW

TOTAL

Unaided Recall 5 4 2 0 11

Aided Recall 3 8 3 14 28

Total Aware 8 12 5 14 39

Eight (0.7% of the 1,018) said that they had visited the website. Another 12 (1% of 1,018) said that they intended to visit the site within the next two weeks, and 5 (0.5% of 1,018) planned to visit the site in the next three months. All other respondents have no plans to visit the site.

Of some interest is that those with unaided recall were more likely than those with aided recall to say that they had visited the site or plan to visit the site in the next two weeks (x2, p = .01). We also found that those in the repeat group were more likely than those in the new group to have visited the site or to have plans to visit the site, although the differences were not statistically significant.



Characteristics of betterbricks.com Visitors

Though a very small group, we examined the characteristics of the eight respondents who reported visiting the website. Table 9 displays some of the characteristics and qualitatively compares them to those of the follow-up sample as a whole.

Table 9: Website Visitors Compared with Follow-Up Sample

CHARACTERISTIC DESCRIPTION OF VISITORS (N=8)

COMPARISON WITH ENTIRE SAMPLE

(N =1,018)

Business Type Office: 7 Health care: 1

Office more prevalent than in sample

Interview Group Previous; 6 New: 2

Previously interviewed more prevalent than in sample

Number of Employees 20 to 49: 4 50 to 99: 1

250 to 500: 2 Over 500: 1

Larger number of employees more prevalent than in sample

Decision-Maker or Influential Staff

Decision-maker: 8 Decision-maker more prevalent than in sample

Age of Building 5 to 20 years: 1 Over 20 years: 7

Older buildings more prevalent than in sample

Square Footage of Building Under 10,000 Sq.ft.: 1 10,000 to 25,000 Sq.ft.: 3 50,000 to 100,000 Sq.ft.: 1

Over 100,000 Sq.ft.: 2

Larger buildings more prevalent than in sample

Making Changes to Building Yes: 7 No: 1

Changes to building m ore prevalent than in sample

Sources Would Use for Improving Productivity in Workspace

Internet: 5 Other sources: 6

Internet more prevalent than in sample

Hours per Week on Internet 2, 4, 5, 6 hours: 1 each 15 hours: 2 20 hours: 2

Average hours: 11

Higher usage more prevalent than in sample



These characteristics are consistent with the goals for the campaign and suggest that the campaign did reach a subset of the target market: office decision-makers in the process of considering a change in their workspace who are frequent users of the Internet.

Response to the Website

Of the eight respondents shown in Table 9 to report having visited the betterbricks.com site, six respondents agreed with the statement that “I believe what they are saying” in the website. All six intended to use the information presented on the website. Their responses to the site were not predicted by their responses to the ad, as only two of the six respondents had said they were very interested in what the ad was saying, while four had said they were somewhat interested. The six site visitors comprised two newly-interviewed respondents and four previously-interviewed respondents.

Two of the eight respondents who reported visiting the site (both in the repeat group) reported either that they did not recall enough about the site to voice an opinion or that they didn’t believe the information on the site.

PHYSICAL WORKSPACE AND EMPLOYEE PRODUCTIVITY

In addition to awareness of the RPIP campaign, the RPIP campaign seeks to change attitudes and knowledge about energy efficient building practices. The follow-up and baseline surveys therefore ask questions about the effect of the physical workspace on employee productivity.

Just over 60% of both groups thought that the physical workspace had a large effect on productivity (see Table 10). However, previously interviewed respondents were significantly more likely than newly interviewed respondents to rate the effect of the physical workspace as “large” (Pearson x2, p = .065; Likelihood ratio, p = .041).



Table 10: Effect of Physical Workspace on Employee Productivity

EFFECT OF PHYSICAL WORKSPACE ON PRODUCTIVITY INTERVIEW GROUP

LARGE EFFECT SOME EFFECT NO EFFECT/NOT SURE

Baseline Sample (n=1,037) 61% 36% 3%

Follow-Up Sample (n=1,018) 62% 35% 3%

New Group (n=570) 59% 36% 5%

Repeat Group (n=448) 64% 34% 2%

We solicited top-of-mind responses to the question “When you think about your organization’s physical space, what factors do you feel contribute the most to employee productivity?” As shown in Table 11, there were no differences between the responses of the follow-up and baseline samples. However, follow-up respondents who had been previously interviewed were significantly more likely than newly interviewed respondents to mention lighting and comfortable temperature in their top-of-mind responses (x2, p = .002 and .001, respectively).

Table 11: Top-of-Mind Mention of Workspace Features

FEATURE OF PHYSICAL WORKSPACE INTERVIEW GROUP

GOOD LIGHTING NATURAL LIGHT COMFORTABLE

TEMPERATURE

Baseline Sample (n=1,037) 30% 12% 17%

Follow-Up Sample (n=1,018) 30% 12% 17%

New Group (n=570) 26% 12% 14%

Repeat Group (n=448) 34% 12% 22%

Not Aware of Campaign (n=965) 31% 12% 17%

Aware of Campaign (n=53) 36% 15% 28%



An indication that the campaign did inform respondents is found in comparison of the top-of-mind mentions by those were aware of the campaign to those who were not. Respondents who were aware of the campaign were more likely to mention each of the three factors, although the difference between the aware and unaware groups was only significant for comfortable temperature (x2, p =.03).

As we found in the baseline study, respondents who said that the physical workspace had a large effect on employee productivity were more likely to give top-of-mind mention of lighting (33% compared with 24%, x2, p = .000) and natural light (14% compared with 8%, x2, p = .06) as physical factors contributing the most to productivity. In addition, as we found in the baseline study, respondents from offices and schools were more likely than other respondents to include all three factors in their top-of-mind responses, a difference that was significant for natural light (14% compared with 9%, x2, p = .02). Respondents from sites with 100 or more employees were also more likely than other respondents to mention lighting, natural lighting, or comfortable temperature in their top-of-mind responses, a difference that was significant for lighting (38% compared with 28%, x2, p = .005).

For respondents who did not identify lighting, natural light, or comfortable temperature in their top-of-mind thoughts on the physical factors that contribute most to productivity, we asked them to rate the effect of these three factors. Table 12 describes the proportion that rated each factor as being of large, moderate, or small importance, and itemizes the previous top-of-mind responses as a subset of those who consider the factors to have large effects on productivity. Differences between the baseline and follow-up responses were statistically significant.



Table 12: Perceived Importance of Workspace Features on Employee Productivity

FOLLOW-UP (N=1,018)

RATED IMPORTANCE ON EMPLOYEE PRODUCTIVITY

BASELINE (N=1,037)

ENTIRE SAMPLE (N=1,018)

NEW GROUP (N=570)


GOOD LIGHTING SYSTEM

Large Importance 89% 91% 89% 93%

• Included in “Top of Mind” Answer 19% 31% 28% 35%

• Prompted Response 70% 60% 61% 58%

Moderate Importance 10% 8% 10% 7%

Small Importance or Don’t Know 1% 1% 1% 0%

NATURAL LIGHT IN WORKSPACE






GOOD TEMPERATURE






For the follow-up survey we asked respondents about the importance to productivity of their ability to control workspace features. One-third of respondents thought that it was very important to employee productivity for employees to be able to control the lighting and the temperature in their own workspace (see Table 13).



Table 13: Importance of Control of Workspace Features on Productivity (n=1,018)

RATING LIGHTING TEMPERATURE

Very Important 33% 33%

Somewhat Important 50% 50%

Not Important/Not Sure 17% 17%

In both the follow-up and baseline surveys we asked respondents to consider the workspace features they had mentioned as important for employee productivity and to tell what steps they had taken to increase productivity in the workspace. Just a little more than 20% of respondents had not taken any steps or could not identify any steps that they had taken. The remaining nearly 80% of respondents reported taking one or more actions. Table 14 gives the actions they had taken as a percent of respondents mentioning actions, providing a comparison of the follow-up and baseline responses.

Follow-up respondents have taken significantly different steps to improve productivity than baseline respondents (x2, p=.0000). Respondents in the follow-up study reported taking steps to get better lighting (including natural light) in the workspace more frequently than baseline respondents (21% compared with 10%). The same pattern was true for steps to improve heating, cooling, or ventilation (12% of the follow-up sample compared with 7% of the baseline respondents). Furthermore, follow-up respondents were less likely than baseline respondents to identify steps taken to improve productivity that related to non-physical factors, such as management, communication, compensation, and issues among co-workers.



Table 14: Steps Taken to Increase Productivity (Multiple Responses Permitted; Tallies Exceed 100%)

STEPS TAKEN TO INCREASE PRODUCTIVITY BASELINE (N=805)

FOLLOW-UP (N=788)

Reorganized Layout of Workspace, Remodeling 41% 46%

Technology, Equipment, Supplies Upgraded 20% 17%

Ergonomic Features 12% 15%

Better Lighting, Natural Lighting 10% 21%

Improved Heating, Cooling, Air Quality 7% 12%

New Building, Expanded Facilities 5% 10%

Aesthetic Appeal of Workspace, Safety 6% 8%

Amenity Space, Meeting Rooms 3% 3%

Personalized Space, Privacy 2% 5%

Noise Reduction Measures 1% 2%

Non-Physical Factors (Management, Communication, Compensation, Co-Workers)

36% 23%

INFORMATION SOURCES

Follow-up respondents gave answers similar to those given by baseline respondents when asked where they would go to find out more about how to improve productivity in their workplaces. About 20% of both the baseline and follow-up samples said that they did not know where they would go. Table 15 provides the percentage of those who gave actual responses, excluding the 20% who didn’t know.



Table 15: Information Sources to Learn More (Multiple Responses Permitted; Tallies Exceed 100%)

INFORMATION SOURCES BASELINE (N=794)

FOLLOW-UP (N=835)

Internal Staff 27% 31%

Reference Materials: 31% 31%

• Journals, Magazines, Newspapers 7% 7%

• Internet 17% 18%

• Books, Library, University 7% 6%

Professional Organizations, Colleagues, Seminars: 18% 13%

• Seminars, Conferences 5% 4%

• Colleagues Not in My Company 8% 6%

• Own Industry’s Association 5% 3%

Professional Design and Construction Firms: 17% 17%

• Architecture Firm 4% 3%

• Workplace Consulting Company 12% 12%

• Engineering or Construction Firm 1% 2%

Regulatory Agency (Local, State, Federal), OSHA, Labor Board

4% 3%

Office Equipment Company 2% 3%

Own Assessment 1% 1%

Other 1% 1%

USAGE PATTERNS OF PUBLICATIONS AND THE INTERNET

We asked those participating in the follow-up survey to tell us what publications they read on a regular basis. Table 16 presents the findings.



Table 16: Publications Read on a Regular Basis (Multiple Responses Permitted; Tallies Exceed 100%)

TYPE OF PUBLICATION ENTIRE FOLLOW-UP SAMPLE (N=1,018)

THOSE MAKING CHANGES TO WORKSPACE

(N=393)

THOSE NOT MAKING

CHANGES TO WORKSPACE

(N=625)

Local or Regional Newspapers 82% 85% 80%

Industry or Professional Trade Publications 66% 68% 65%

National Business Magazines 26% 30% 24%

Business Weeklies 21% 24% 18%

Regional Monthly Business Magazines 21% 23% 20%

Most respondents (82%) read local or regional newspapers on a regular basis. Two-thirds of the sample also read industry or professional trade publications regularly. Between one-fifth and one-quarter of the sample read business weeklies, business monthly magazines, or national business magazines. Just over 5% of respondents reported that they read none of these publications on a regular basis. Those making changes to their workspace were significantly more likely than those not making changes to read local newspapers, business weeklies, and national business magazines (t-test, p < .05).

Nearly four out of five follow-up respondents (79%) have access to the Internet at home; 90% have access at work. The questions soliciting information about Internet access and usage were refined for the follow-up study and thus the responses cannot be directly compared with the baseline data. In the baseline study (which explored work use of, but not work access to, the Internet), 72% of decision-makers and influential staff reported using the Internet at work.

In the follow-up survey, we asked respondents that had access to the Internet how many hours per week they used it at work and at home, combined. Their responses are grouped into categories and presented in Table 17. The newly and previously interviewed groups differ significantly (x2, p=.04).



Table 17: Hours Per Week of Internet Usage (n=962)

HOURS OF INTERNET USE PER WEEK

FOLLOW-UP SAMPLE WITH INTERNET

(N=962)

NEW GROUP (N=538)


1 Hour or Less 24% 21% 27%

2 to 4 Hours 29% 28% 31%

5 Hours 12% 13.6% 11%

6 to 10 Hours 18% 18% 18%

11 to 15 Hours 7% 8% 6%

16 to 20 Hours 5% 5% 5%

Over 20 Hours 5% 7% 3%

MEAN 7 hours 8 hours 6 hours

The means for each group are also shown in Table 17. Respondents having access to the Internet used the Internet at work and at home for about seven hours a week. The difference in use between the newly and previously interviewed groups was significant. The new group reported using the Internet an average of eight hours a week, compared with six hours for the repeat group (t-test, p = .001).

This finding is noteworthy because it provides further evidence of the effect of priming that occurred during the baseline survey. All other things being equal, one would expect those who use the Internet more hours (the new group) to be more likely to notice and respond to a campaign for an Internet brand. Yet, it was the group that used the Internet less, but had been exposed during the baseline survey to the idea that productivity and workspace could be linked that responded more to the brand campaign.

SUMMARY OF AD CAMPAIGN AWARENESS

As summarized in Table 18, just under 4% of the entire follow-up sample was aware of the ad campaign and the betterbricks.com name. Less than 1% of the entire sample visited the website after viewing an ad. For those newly surveyed – who were not primed by the baseline survey to be aware of the topic of productivity in the workspace – just over 2% were aware of the campaign and about one-half of one percent visited the website after viewing an ad.



Table 18: Summary of betterbricks.com Awareness (n=1,018)

LEVEL OF AWARENESS NEW GROUP

REPEAT GROUP

TOTAL FOLLOW-UP

SAMPLE

Aware of betterbricks.com 12 (2.1%) 27 (6.0%) 39 (3.8%)

Visited Website after Viewing Ad 2 (0.4%) 6 (1.3%) 8 (0.8%)

Not Aware of betterbricks.com 558 (97.9%) 421 (94.0%) 979 (96.2%)

Total 570 (100%) 448 (100%) 1,018 (100%)


CHAPTER 4: ARCHITECT ASSESSMENTS FOR THE RPIP

INTRODUCTION

The RPIP campaign, through the Advocacy Program, intended to target supply-side providers of design and engineering services during the summer of 2000. The outreach was to include Design Guidelines on practices discussed at the betterbricks.com website and targeted presentations throughout the region featuring leading designers and engineers who would promote betterbricks.com to the supply-side service providers.

The development of the Design Guidelines took more time than expected (see Chapter 9, New Buildings Institute discussion) and they were not available in 2000. Without branded material, the outreach to supply-side providers was limited to placement of print ads in Architectural Record in seven issues between May 2000 and March 2001. With such limited outreach, a specific survey targeted to supply-side service providers did not make sense. In addition, Research Into Action, Inc. was implementing an evaluation of the Alliance-funded Architecture + Energy (A+E) program during spring 2001, creating the potential for including questions pertinent to the RPIP evaluation. The Alliance Evaluation Manager agreed to include questions to track client requests for energy efficiency services and awareness of the RPIP betterbricks.com campaign in the A+E evaluation survey.

METHODOLOGY

Survey Design

The approach of using the A+E evaluation survey for RPIP follow-up enables us to provide a timely response to the campaign and avoids over-surveying architects. However, there are also limitations to this approach. The survey could not fully replicate the baseline survey due to its dual purposes. Thus the questions about client requests for services are different. In addition, the survey includes both participants in A+E and non-participants, thus necessitating a weighting scheme to ensure the A+E participants do not bias the results.

Despite these limitations, there is strength to this approach. First, the A+E evaluation sample was drawn from all architectural firms in the region with information on size-of-firm, while the RPIP supply-side baseline focused on

Chapter 4: Supply-Side Assessments for the RPIP – Architects


architects who are members of the American Institute of Architects (AIA). Second, the survey developed for the A+E evaluation is designed to provide measurable levels of implementation of energy efficiency solutions in design practice, we believe this may provide a better baseline for tracking future impacts from RPIP efforts than the original baseline. Third, the baseline was conducted in spring 2000; since that time, an energy crisis has occurred on the West Coast, leading to requests from the region’s governors and power companies for businesses and consumers to conserve energy at every opportunity. This external factor would be difficult to account for without more explicit measurement, such as was developed for the A+E evaluation survey. We believe that this will be easier to account for in future follow-ups to RPIP using the A+E survey design, rather than the baseline. A copy of the survey is provided in Appendix B.

Sample

We purchased a business list of architectural firms from InfoUSA. The list for the Pacific Northwest has about 1,800 firms. Of the 1,800 firms, 236 have ten or more employees.

We created a master list of firms from four sources: the InfoUSA list, firms listed in the most recent AIA member database, firms that participated in A+E, and firms that Research Into Action, Inc. has interviewed. The master list indicated the source or sources on which each firm name appears.

We drew two samples: participating firms (those with staff that participated in A+E during the Alliance-funded period) and nonparticipating firms (those without such staff). For the participating firms, we attempted to identify a participating individual and a nonparticipating individual to survey. However, participants were not willing to volunteer non-participants, so this effort was not continued after the first few attempts.

For non-participants we stratified the list by firm size to match the participant list. We randomly selected firms to call in each size category and asked to speak to a principal or project director. In order to ensure inclusion of design/build firms, in addition to the InfoUSA and AIA lists we also contacted general contracting companies and developers and identified those with architects on staff. Table 19 displays the results of the sampling strategy. This strategy was then used to develop the weights for each stratum.



Table 19: Sample for Survey

NO. EMPLOYEES/

FIRM

POPULATION OF A+E

PARTICIPANT FIRMS

COMPLETED INTERVIEWS

POPULATION OF A+E NON-

PARTICIPANT FIRMS

COMPLETED INTERVIEWS

POPULATION OF NON-

RESIDENTIAL ARCHITEC-TURE FIRMS

TOTAL COMPLETE

INTERVIEWS

>50 17 11 18 12 35 23

20-50 19 14 52 12 71 26

10-19 11 6 90 11 101 17

5-9 18 7 240 10 158 17

1-4 ~40a 5 ~140b 5 ~180 10

Total 105 43 540 50 545 93

a. The part icipant and non-participant populations include many small firms that primarily do residential design. These firms were screened out during calls to request interviews and the proportion of all firms in the population doing nonresidential was estimated from the calls that were made.

b. The number of non-participants in the size range of 1-4 are only those from the InfoUSA database, not the AIA directory, since the AIA directory lists individuals. Nor does the total include firms identified with architects who do general construction or development. Furthermore, the total number of firms was adjusted to estimate those doing only nonresidential design.

FINDINGS

The follow-up survey focused on two main questions: whether client interest in energy efficiency design solutions had increased in 2000, and whether architects were aware of the RPIP betterbricks.com campaign and, if aware, were they using it.

Client Interest in Energy Efficient Design Solutions

In the follow-up survey we asked architects to indicate whether discussions with clients about energy efficiency were common, or whether such discussions presented a barrier with clients. Forty-four percent responded that, on a five-point scale, this is a barrier or a major barrier. We also asked architects whether clients had been asking for energy efficiency solutions more in the past three months. Forty-two percent of the respondents indicated that their clients had been asking for energy solutions more frequently in the past three months; another 10% noted that there had been discussions, but no specific requests.



In the baseline survey with architects, we asked the architects to rate on a scale of one-to-five how often their clients asked them to address different design criteria in projects. The design criteria we asked about were those targeted by the betterbricks.com website. As can be seen in Table 20, for most of the design criteria, over 50% of the architects reported that clients were requesting they address the criteria prior to the launch of the RPIP betterbricks.com campaign.

Table 20: Baseline Percent of Time Design Criteria Are Addressed*

DESIGN CRITERIA ARCHITECTS (N=54)

CRITERIA RELATED TO TECHNICAL ENVELOPE AND OVERALL SYSTEMS

Good Temperature Control 85%

Low or Reduced Operating Costs 59%

Natural Light in Workspace 57%

Energy Efficiency 59%

Meeting Company Environmental Goals 28%

CRITERIA RELATED TO AESTHETICS OR INTERIOR WORKSPACE DESIGN

Improved “Look” for Customers or Clients 76%

Quality Lighting 72%

Room for Future Staff Expansion 69%

Employees’ Physical Comfort 74%

More Space for Existing Staff or Customers 54%

More Storage or Warehouse Space 54%

Ways to Increase Worker Satisfaction through the Design 39%

Ways to Increase Worker Productivity 46%

Room for More Product Display 20%

Design Considerations to Reduce Worker Turnover 11%

* Percent of respondents giving a 4 or 5 rating on a five-point scale where 5 signified “all the time” and 1 signified “never.”



With such a high level of requests from clients to address these design criteria, a more precise description of design solutions seems warranted and might lead to a different assessment of the level of implementation. In the follow-up survey questions we focused on specific energy efficient design solutions and asked whether during the past year clients had requested the architect address these in their projects. As shown in Table 21, for all but one of the design solutions, less than 40% of the architects reported clients asking for these design solutions in the past year.6

Table 21: Whether Clients Requested Energy Efficiency Design Solutions in 2000

DID CLIENTS REQUEST IN THE LAST YEAR? PERCENT* (N=93)

Pre-Design Discussion of Energy and Resource Savings Opportunities 57

Site Orientation Issues 25

Building Envelope to Reduce Energy Requirements 34

Daylighting Features 36

Passive Systems to Augment Electromechanical 15

Lighting Efficiency 38

HVAC Efficiency 35

Water Heating Options 17

Lifecycle Costing Analysis 39

Energy Use Modeling 22

* Percent reflects weighting of the sample by size to represent the population.

Table 22 shows the number and percent of architects reporting that they implemented these design solutions in their projects over the past year. As with the betterbricks.com baseline study, the follow-up survey indicates a substantial amount of energy efficient design practice is already occurring in the region. Betterbricks.com sought to increase client requests for energy efficient design

6 One drawback of the A+E survey approach for measurement of RPIP effectiveness was the need to use the

term "energy efficient" to describe some of the design solutions. The RPIP campaign explicitly sought to avoid the term energy efficiency, while the A+E evaluation explicitly targeted these actions.



solutions. Comparing Tables 21 and 22 it appears that in 2000, architects were more apt to be making the decisions to implement energy efficiency solutions than responding to client requests. In comparison to the baseline survey, it appears that the A+E questions may result in a slightly finer discrimination of design practice. However, it is important to note that comparing Tables 21 and 22 is not directly possible, so that we cannot assess whether there was a change in client requests in 2000. To directly compare future design practice to the baseline period 1999, we would recommend using both the baseline questions and the A+E questions.

Table 22: Whether Architects Implemented Energy Efficiency Design Solutions in 2000

WHETHER PROJECTS INCLUDED SOLUTIONS IN PAST YEAR PERCENT* (N=93)

Pre-Design Discussion of Energy and Resource Savings Opportunities 81

Site Orientation Issues 61

Building Envelope to Reduce Energy Requirements 94

Daylighting Features 88

Passive Systems to Augment Electromechanical 60

Lighting Efficiency 79

HVAC Efficiency 79

Water Heating Options 43

Lifecycle Costing Analysis 73

Energy Use Modeling 64


Awareness of the RPIP betterbricks.com Campaign

A significant question for the follow-up survey was an assessment of architect awareness of the RPIP betterbricks.com campaign. The A+E participants and architects in larger firms were more likely to be aware of the campaign. We identified 19 respondents who recalled the campaign, weighting their responses for size of firm and participation in A+E; the resulting weighted recall for all designers



of the RPIP betterbricks.com campaign is 9.9%. The unweighted awareness responses for the participants and non-participants by size of firms are presented in Table 23.

Table 23: Architect Awareness of RPIP betterbricks.com Campaign by Strata

FIRM STRATA A+E PARTICIPANTS

AWARE BY TOTAL IN STRATA

PERCENT OF STRATA

NON-PARTICIPANTS

AWARE BY TOTAL IN STRATA

PERCENT OF STRATA

Large Firm (50+ Designers) 6/11 55 2/12 17

Medium Large (20-49 Designers)

6/14 43 0/12 0

Medium (10-19 Designers) 2/6 33 1/11 9

Medium Small (5-9 Designers) 1/7 14 1/10 10

Small (1-4 Designers) 0/5 0 0/5 0

A colleague was the source of information for 47% of the 19 architects. Yet, when source of information on betterbricks.com is weighted for size of firm and participation in A+E, Table 24 shows that TV advertisements and print had the largest effect on the architects. TV advertisement was the source for 4% of the architects (a little higher when combination sources are included), which is approximately the same as for business decision-makers and influential staff. However, the additional awareness for architects came from print advertisements, colleagues, and experience with the Alliance.

We also asked the architects whether they had visited the betterbricks.com website or intend to do so in the future. Weighted for size of firm and participation in A+E, 8% of the architects report that they have visited the website, 2% say that they plan to use it in their design practice and 2% report they will revisit the site. An additional 5% report that they intend to visit the website in the two weeks after the survey and 1% in the next three months.



Table 24: Source of Information on betterbricks.com

SOURCE OF INFORMATION PERCENT* (N=93)

Colleague 3

TV Advertisement 4

Print Ad 2

Alliance Activity <1

Combination <1

Not Aware 90

Total 100


SUMMARY OF FOLLOW-UP WITH ARCHITECTS

Sampling from architects throughout the region, we found that client requests for different energy efficiency features are typically made less than 40% of the time. Architects, on the other hand, report including most energy efficiency features well over 75% of the time. At 9.9%, awareness of the RPIP betterbricks.com campaign was higher for architects than for business decision-makers and influential staff. Notably, TV advertisements accounted for about 4% of that awareness, comparable to the total awareness achieved for business decision-makers and influential staff. Architects awareness was higher due to the influence of colleagues, print advertisements, and experience with Alliance projects.


CHAPTER 5: BETTERBRICKS.COM ADVERTISING FOCUS GROUPS

INTRODUCTION AND GOALS

While the central product for the RPIP campaign is the betterbricks.com website, the prime drivers to get people to that website have been television and print advertising. As described in Chapter 1, the Alliance sponsored considerable up-front qualitative research with business people and supply-side actors to develop the underlying logic of the advertising campaign, which linked productivity with building efficiency improvements.

The Alliance’s advertising contractor, Cole & Weber, then translated that logic into advertising media. Due to time constraints and other factors, the advertisements were not tested with target audiences before being launched. Thus, after the campaign launch, and as part of our evaluation of the RPIP effort, we conducted focus groups with three target audiences to:

Ø Find out more about the roles of those involved in determining the physical workspace and environment, and discovering their perspectives on what is important in those decisions.

Ø Obtain feedback on the effectiveness of the TV and print advertising, including how well target audiences understood the advertising messages, what they thought about the ads, what feelings the ads evoked, and what they thought the ads were trying to get them to do.

Ø Find out more about the sources target audiences use to get information on employee productivity and the physical workspace, and to discover the best ways to reach them with information on those topics.

Ø Gather advice about how to improve betterbricks.com efforts.

METHODS

We selected three key audiences for these focus groups, representing both demand-side and supply-side actors:

Chapter 5: Betterbricks.com Advertising Focus Groups


Ø Decision Makers and Influentials – because they may have immediate opportunities to decide on or influence physical workspace decisions. We split this pool into two groups:

• Movers (Group 1: 9 participants) – Decision-Makers/ Influentials who had recently moved or were in the process of moving; and

• Non-Movers (Group 2: 8 participants) – Decision-Makers/ Influentials who had not recently moved and were not contemplating a move.

Ø Developers/Real Estate Brokers (Group 3: 9 participants) – because they are key actors in determining the future of new commercial building design through their contacts with tenants and owners.

The focus groups were held in the Portland area in mid-October 2000, with a total of 26 participants attending (as indicated in parentheses above). Each group was led by a trained facilitator, lasted approximately one-and-one-half hours, and moved through a series of similar questions designed to address the goals listed in the introduction to this chapter. In general, the groups covered these topics:

1. Roles and perceptions about workspace design issues and trends

2. Reactions to television advertising

3. Reactions to print advertising

4. Characterization of the betterbricks.com brand identity

5. Information sources

In the next section we discuss the key findings and implications for each topic area.

KEY FINDINGS AND IMPLICATIONS

The focus group analysis revealed several major areas of insight about target audience response to, and perspectives about the betterbricks.com advertising campaign and the underlying mission of the RPIP. This section summarizes the key findings and implications from this research. We have added quotes from focus group participants where they add value to the findings.



Roles and Perceptions About Workspace Design Issues and Trends

How Target Audiences Connect the Physical Workspace to Productivity

Key Findings: “When an employee is in a comfortable area, they work better, and especially if it’s ergonomically correct, it’s efficient for them, they produce more.” – Decision-Maker

Consistent with findings from other RPIP evaluation efforts, participants in all three focus groups realize that workspace relates to productivity. However, all three focus groups associate physical workspace productivity with topics such as layout, equipment, ergonomics, ambiance, and accessibility rather than with energy efficiency (infrastructure) components such as lighting, air quality, and temperature.

Notably, decision-makers and influentials in the non-mover group did mention energy-related workspace components, while the mover group did not. This leads us to speculate that movers were too involved with “space wars” to be able to think about workspace infrastructure changes. On the other hand, non-movers were thinking more about the aspects of their current work environment that needed improvements, like lighting and temperature.

Implications: The link between energy efficiency improvements and productivity is not likely to occur or rise to the top of people’s minds unless the connection is strengthened.

Trends in New Building Design

Key Findings: “ . . .the two most important things we are asked about are comfort in the space and employee amenities.” – Senior Property Manager

“… and energy savings are big issues, because we are seeing significant increases in the cost of utility pricing (most nod in agreement).” – Real Estate Broker

Trends in building design for Class A buildings, as reported by the developer/real estate focus group participants, tend to be consistent with one another and quality oriented; they include ambiance, amenities, functionality, flexibility, communications sophistication, and environmental concerns such as indoor air quality and energy efficiency. When specifically asked if they saw a trend toward greater concern with utility costs, 6 of 9 said “yes.”



Implications: Those who have their ears to the ground about trends for new commercial buildings cite factors that are consistent with RPIP concerns. This consistency can be used to the program’s advantage.

Key Players and Timing Issues in the Building Design Process

Key Findings: “With me, it’s our owner. He has the checkbook, so he has the greatest influence. It would also come down to targeted tenants . . . I would hope the owner sits down with the architect and tenants . . .” – Commercial Leasing Representative

“The best way for new ideas to get incorporated into building design are for the developers and tenants to work together.” – Retail Leasing Agent

The developer/real estate broker focus group identified an array of equally essential, important, and interactive players who most influence Class A commercial building design. These players include building tenants and users, architects, developers and owners, and regulating jurisdictions. Although these are the terms most used for the players involved, individuals have a variety of job titles and a range of responsibilities depending on the circumstances. For instance, the “users” could be the customers in a retail space or the workers in an office building, while the building owner could also be the developer. Participants in all three groups also mentioned that efforts to influence energy efficiency improvements must occur early in the design process.

Implications: The RPIP strategy to influence energy efficiency design considerations will not be effective unless it reaches all of the key players early in the design process.

Reactions To Television Advertising

Key Findings: “It’s all style and no substance.” – Developer

“I have to admit, that’s one of my favorite ads.” – Developer “But do you know what it wants you to do?” – Group Facilitator “Oh no.” – Developer

About half of those attending the focus groups had seen each of the two betterbricks.com TV ads before coming to the focus groups, indicating that the target audience noticed the ads. Notably, more of the developer group (7 out of 9) had seen the ads before attending.



There were mixed reviews about how much they liked the ads. Most participants understood that the ads were related to workspace issues, and some identified that they were related to productivity. Initially, about one-third of participants overall understood that the ads were asking them to visit a website. (This finding was based on individual written responses after participants watched the ads.) Following more group discussion, about half of the participants said they thought the ads wanted them to visit the website.

When asked if they understood the specific intent, services, target audiences, or sponsorship of the ads, most participants said the ads were indirect and unclear, and that they were uncertain or confused after seeing the first two 30-second spots. However, after seeing the third spot (10 seconds), which told and showed them the name of the website, and told them to go to the website to find out more about workspace improvements, they responded more positively to the overall TV campaign.

Implications: The television ads were able to get the attention of target audiences, but they were less effective at informing and convincing participants to go to the website.

Reactions To Print Advertising

Key Findings: None of the 26 focus group participants reported they had seen any of the print ads, suggesting that the impact of these ads has been minimal. However, the decision-makers and influentials responded much more positively to these ads, especially those in the non-movers group. Non-movers liked that the ads raised really important “infrastructure” issues (like lighting and temperature control) that decision-makers don’t normally connect to workspace planning. They also said the situations displayed in the ads were familiar problems to them. Developers were less positive about the print ads, but did acknowledge later in the discussion that the topics of lighting and heating were very important to them.

During the discussion of the print ads, several decision-makers and influentials voiced concerns about the strategy of trying to motivate general employees to agitate for workspace changes. Some people thought the advertising could produce more “whining” and less work.

Implications: Print advertising does allow more content detail to be presented to target audiences. Consistent with other findings, once the content of betterbricks.com was made clear, target audiences developed a greater appreciation for its intent, purpose, and products.



Characterization of the betterbricks.com Brand Identity

Key Findings: When asked to choose adjectives to describe the betterbricks.com brand, the most common adjectives chosen were “forward thinking” and “high tech” among movers and non-movers, and “trendy” and “clueless” among the developer/ real estate broker group. Although in some respects the two target audiences chose similar adjectives, the connotations for “trendy” among those in the developer/real estate broker group tended to be negative, while the connotations for “forward-thinking” among decision-makers and influentials tended to be positive.

After being shown a description of the betterbricks.com service, most focus group participants were completely surprised the effort was sponsored by a not-for-profit agency. Among movers and non-movers, this was a definite positive, and they particularly appreciated the free consultation from a technical advisor. Among developers/real estate brokers, however, the not-for-profit status was immaterial and there was little interest in what they considered a small amount of free consulting, since most of these individuals work with large-scale projects. (Even though the betterbricks.com description given in the focus group specified one hour of free consulting instead of the four hours actually provided, it is likely developers would still have felt this was inadequate. They often felt they had other, better, and more familiar places to get information, such as from utilities and other contractors.)

Implications: The very different reactions between supply and demand-side target audiences to the attributes of the brand, to betterbricks.com’s sponsorship, and to the services provided, shows the difficulty in finding a single approach that will work for all target groups.

Information Sources

Key Findings: Both movers and non-movers reported they most often go to supply-side resources – architects and engineers – when they need expertise about making workspace improvements. Some also mentioned trade publications (specific to their industries) and the Internet in general (although no specific websites were mentioned). The developer/real estate group also reported they would use supply-side expertise; they seemed to be especially oriented to personal relationships.

Implications: This finding further supports the importance of equal supply-side involvement in the RPIP.



RECOMMENDATIONS

From the Focus Group input we make the following recommendations:

1. The link between productivity and energy efficiency needs to be direct, highlighted, and justified in informational, public relations, and advertising efforts. The RPIP can then take advantage of the fact that decision-makers, influential employees, and developers/ real estate brokers are already concerned about productivity.

2. The RPIP should develop targeted outreach materials, make direct contacts, and offer partnership opportunities for real estate developers and brokers based upon shared visions of the future of commercial buildings. Supply-side actors can play an important role in promoting energy efficient construction. The most effective collaborations for innovation will be with developers and brokers for Class A buildings, where owners and tenants are willing to pay for quality improvements.

3. If mass media TV advertising is used, it needs to clearly convey the links between productivity, energy efficiency, and the benefits of visiting the website. When directing viewers to the website, it will be more effective to include both a verbal and visual instruction (i.e., voice plus displaying the address).

4. If print advertising is used, it needs to be better placed in trade publications targeted to specific types of businesses (e.g., hospitals or schools) and it needs to be run more frequently.

5. To reach both supply and demand-side audiences, identify and use “hooks” that interest and compel both groups. For instance, participants in these groups voiced strong shared interests in solving the real problems of lighting, air quality, and temperature.

6. Where supply and demand-side groups view attributes of betterbricks.com differently, determine how to tailor the emphasis or message for each group so that the campaigns complement one another. For instance, demand-side groups tended to be admiring of the “fast company” image (a trendy company on the cutting business edge) but the supply-side group was somewhat distrustful and offended by it. In another instance, participants in the demand-side groups tended to favor betterbricks.com being sponsored by a not-for-profit agency, but supply-



siders were more neutral or negative. Further research may be needed to determine how these potentially conflicting views of the product can be effective with both groups.

7. The information and technical consulting services should be made more useful to the supply-siders, since they are likely to need even more in-depth services than those on the demand-side. Review of existing, and perhaps future, market research with these groups, will be needed to devise the right approaches.

8. Supply-side actors need to be an equal target to demand-side actors, but, as described above, the campaigns need to be both complementary and tailored to specific audiences.

9. Only the developer/real estate group provided some substantial “final advice,” for betterbricks.com, but it would probably be echoed in the demand-side groups:

Ø “Keep it simple and clear.”

Ø “Coordinate your service with other related services.”

Ø “Let me know what’s in it for me.”


CHAPTER 6: BETTERBRICKS.COM WEBSITE USABILITY TESTING AND USE

INTRODUCTION

The RPIP betterbricks.com website was launched May 1, 2000. The advertising campaign directed web traffic to the betterbricks.com website. In this chapter we:

Ø Present the results from the website usability tests conducted in August 2000;

Ø Summarize the results of an analysis of website traffic between May 1, 2000, and March 25, 2001; and

Ø Report on use between May 1, 2000, and March 1, 2001, of the Advisor Service that can be accessed through the website.

RPIP’S BETTERBRICKS.COM WEBSITE USABILITY TEST

Introduction and Goals

The RPIP effort in 2000 resulted in the launch of the betterbricks.com brand awareness campaign and the betterbricks.com website. The website is the campaign’s central information product to inform businesses about the worker benefits of high efficiency choices in commercial buildings. The betterbricks.com television and print advertising, as well as public relations efforts, all encourage audiences to visit the website. Visitors to the website can also contact an advisory service that provides further information and referrals. Upon request, a free, four-hour consultation on specific workplace energy practices can be arranged.

When the website was developed, usability testing with potential users was not conducted, primarily due to a lack of time as a result of the desire to launch the site simultaneously with the media campaign, which was targeted for late spring. Usability testing, which is usually qualitative, examines how visitors to a site actually behave when using a website and focuses on their requirements. Concern about how users would respond to the website led the evaluation team to propose conducting a usability test. In addition, after the launch of the website, a number of users noted enough difficulties and “bugs” in the site that the Alliance decided to sponsor usability testing of betterbricks.com V1.0 to look for:

Chapter 6: Betterbricks.com Website Usability Testing and Use


Ø Misconceptions or confusions about the purpose of the site;

Ø Task-performance and navigational problems; and

Ø Items or components visitors particularly liked or disliked.

The usability testing was completed and reported on in the fall of 2000. The Alliance asked Cole & Weber, the website developer, to implement many of these recommendations and conduct usability tests of the new version, which they subsequently did. The betterbricks.com V2.0 was launched in late January 2001.

Methods

Hypercerulean, Inc, a Seattle firm that specializes in website usability studies, along with Alliance evaluation staff and contractors, designed the test procedures. Hypercerulean conducted the tests in two locations – Seattle, Washington, and Boise, Idaho – in late August 2000. Participants were recruited to match various target audiences of the website: general commercial employees, influential staff who strongly influence space decisions, decision-makers who make the final decisions about space, architects, and developers. In all, 17 people from the website’s target audiences participated in one-and-one-half-hour in-depth individual interviews, as shown in Table 25 below.7

Table 25: Distribution of Usability Test Participants by Location and Target Audience

TARGET AUDIENCE SEATTLE BOISE TOTAL

General Commercial Employees 2 2 4

Influential Employees 3 2 5

Decision Makers 2 1 3

Architects 2 1 3

Developers 1 1 2

Total 10 7 17

7 Standard sample sizes for usability testing range from 8-10 per test. The sample of 17 resulted from an attempt

to conduct usability testing with 8 people in two geographic locations – Seattle, Washington, and Boise, Idaho.



The testing proceeded in three phases:

Ø Phase 1 – First Impressions: After seeing two betterbricks.com TV ads, participants were asked to behave as if they were visiting the site by themselves in real life. They explored the site, unguided, for about five minutes and described aloud to the interviewer what they were doing and thinking.

Ø Phase 2 – Tasks: Participants further explored the site, initially without guidance, and were asked to continue to report their behaviors and reactions. Then, after being shown relevant betterbricks.com print ads, they were asked to gather information from the site to convince colleagues of the importance of the physical work environment. Finally, participants were asked to visit specific site locations or features (if they hadn’t already) and then asked for their reactions.

Ø Phase 3 – Follow-Up Questions: After spending an hour at the site, participants were asked follow-up questions about a range of website impressions and issues.

Overall Usability Results

The usability test of V1.0 identified many opportunities to improve and enhance the website. Since prior testing had not occurred, this was not surprising. In all, the usability test identified over 130 opportunities to enhance usability of the site. Participants were especially frustrated by not knowing:

Ø What the website was supposed to do

Ø What information the site offered

Ø Where to find desired information on the site

However, as they became more familiar with the site, most liked it more (12 of 17), and 11 reported that they would recommend it to coworkers, managers, or clients. Participants generally found the content useful, once they understood more about the site’s purpose, once they discovered more of its content, and once they learned how to navigate around it.

Still, it is clear participants want the site to carry more in-depth information in the future, citing needs for information about: how to evaluate workspace problems, how to implement changes before and after studies, regulatory issues, and where to



reach the right contractors. By making many of the improvements described in the remaining sections of this chapter, and expanding the content of the betterbricks.com website over time, the site can be quite a useful tool for its target audiences.

Key Findings and Recommendations

The usability testing revealed six major areas where the betterbricks.com website can be improved. In this section we first describe each major area that needs improvement and then present specific key findings, implications, and recommendations related to that area. A copy of the complete usability test results, including questions and specific details of comments, is included in Appendix D.

Clarify Website Purpose and Sponsorship

For most participants, the purpose and the type of “entity” behind the website was unclear. Participants could not identify from the advertising what type of information or services they would find at the website. In addition, the .com extension suggested to many that betterbricks.com is a for-profit organization, rather than an extension of the Alliance.

Key Finding: Based on the TV ads, participants initially tended to think betterbricks.com was related to office furniture or perhaps Internet services. While all of the participants came to understand that the betterbricks.com website related to improving the work environment, none were certain of the services betterbricks.com provided.

Implication: Web visitors want to know quickly and clearly why a website exists. Lack of clarity of the website purpose frustrated participants. It contributed both to a desire to leave the website initially, and to the unwillingness of a third of the participants to recommend the site to others.

Recommendation: Provide a mission statement prominently on the homepage. State who the website is intended for, “who” betterbricks.com is, who supports it, and whether its agency sponsor is a for-profit or not-for-profit firm. As one participant put it: tell visitors “what inspired the company to exist.”

Key Finding: Ten of 17 participants thought betterbricks.com was (sponsored by) a for-profit company, five thought it was (sponsored by) a not-for-profit entity, and



two were unsure. Six preferred it to be not-for-profit, and no one preferred it to be for-profit.

Implication: For a notable minority of participants, finding out that betterbricks.com is sponsored by a not-for-profit agency was an asset, increasing their trust of the organization and diminishing any concerns about competitive issues. The need to use a .org domain was not viewed as necessary, although a clear statement of organizational sponsorship was viewed as necessary.

Recommendation: State on the homepage that betterbricks.com is sponsored by a not-for-profit agency. List any endorsements from professional or trade organizations.

Improve Website Organization and Navigation

None of the participants had a clear idea of where to start or what paths to follow from the homepage. They also did not understand the range of tools and information available or how the parts of the website were connected to one another. The Discover/Believe/Act tabs were intended as major navigational tools on the betterbricks.com website. Yet, participants rarely used these tabs and had considerable difficulties making them work when they did try.

Key Finding: The division of the website into sections Discover, Believe, and Act meant little to participants and did not provide natural divisions for the website’s information. Most participants did not find the list of all field studies and 9 of 17 participants did not find their way to the Act area without prompting.

Implication: Web visitors tend to be in a hurry. Thus, if they do not get oriented quickly, they leave. They want to know what their options are on a site and how to get to them. Ease of access to information also affects their willingness to recommend a site to others. Thus, access to useful information needs to take precedence over looks.

Recommendation: Organize the website in terms of tasks the visitors want to perform and tools for performing those tasks. Provide layers of detail, so that visitors start at a general overview level and can drill down to details as needed. For instance, a menu on the homepage might have entries such as:

Ø Learn More About: A list of topics such as betterbricks.com, Lighting, Heating, Air Quality, Affect of Work Environment on



Health and Worker Performance, and Field Studies with Real Buildings.

Ø Learn How To: A list of topics such as Rate Your Work Space, Improve Your Work Space, Convince Others, Get Free Advice, and Find Companies to Help Improve Your Work Space.

Ø Research Studies

Ø Regulatory Issues for Workspaces

Ø Related Sites

Ø Privacy Statement

Key Finding: Participants said the Discover/Believe/Act tabs and their menu/ sub-menu structures were hard to notice, hard to understand, and hard to use. For instance, unless the mouse is moved very precisely, the submenus change while the participant is in the process of navigating to them from the main menu.

Implication: Difficulties in noticing and understanding the meanings of the tabs did not motivate participants to click on them. Participants became confused and did not appreciate the time and attention it took when they did try to use them.

Recommendation: Replace the structure with a left margin navigational bar, organized according to tasks visitors want to perform (see section above).

Key Finding: None of the 17 participants were familiar with the convention of clicking on the logo in the upper left corner to return to the homepage. To return to the homepage, they repeatedly clicked the “Back” button, which took significantly longer.

Implication: The homepage is the center of operations and web visitors need to be able to get to it quickly. They become frustrated and annoyed when this simple task becomes difficult and time consuming and are less likely to stay on the site or delve deeply into it.

Recommendation: Include a “Home” button on each web page.



Expand Content

Participants generally liked the content on the betterbricks.com website, once they got past problems of organization and navigation. However, the content is mostly introductory and did not give them the information they would need to take action.

Key Finding: Participants raised questions not answered on the site, such as: What companies can they contract with for good workspace support? How can they evaluate their lighting, HVAC, etc.? What can they do short-term? What can small companies do? What are the cost, timeline, and personnel requirements? What are the health, productivity, and financial issues? Is there good “before and after” information available? What are the regulatory issues?

Implication: While the current level of information may get people thinking and started down the path to action, visitors will need more in-depth information to take action. The website cannot remain static; it will need expansion, culling, and upgrading.

Recommendation: As the website evolves, integrate more detailed information. Provide a simple “suggestions” e-mail link to gather information about what visitors want to know more about. Be careful to follow a clean organizational structure, as described above. As more detailed information is added, it should not obscure the high-level overview.

Improve Certain Style Issues

No major stylistic problems were found in this usability test; still, participants did have some complaints and recommendations, as noted below.

Key Finding: Five participants mentioned the fonts were too small in parts of the website. Also, in some places, light lettering is used on a light background.

Implication: Both of these conditions discourage reading of the text.

Recommendation: Provide adequate font size and good contrast for all text. Remember that website developers are often younger, and have better monitors, than the target audience. Test the website informally with older viewers on lower-end monitors.



Key Finding: While the betterbricks.com website is targeted primarily at professionals who are at work, and likely to have high-end computer systems, they may often visit the website from home.

Implication: Home-based computer systems are often lower-end and are slow at loading complex graphics, making betterbricks.com less user-friendly at home. Web visitors need to understand quickly what the site is about and how it can help them. The decision to stay or leave betterbricks.com will be driven much more by the usefulness of the content than by the advanced artwork. Website visitors often decide to leave a site before the homepage has finished loading.

Recommendation: Keep advanced graphics to a minimum and test the website on low-end systems. Consider removing image fade-ins. Use critical space on the homepage to plainly express the purpose of the site and what tools it provides. Use the top for a mission statement and the left margin for a navigational bar. Graphics should be secondary to communicating purpose and capabilities.

Improve the Advisor Service

The potential to receive free, personalized advice was expected to be a major draw for visitors to the betterbricks.com website. In practice, however, participants gave the Advisor Service a lukewarm reception. This appeared to be due to three reasons: first, due to navigational issues many encountered the area before they had visited the rest of the site; second, the area provided unsatisfactory information on what benefits they would receive from the “Advisor;” and three, the form required to gain access to an advisor was intimidating.

Key Finding: When participants reached the Advisor Service as their first stop, they reported that the long user information form was a barrier. It appeared to be very time consuming or they thought it was a requirement before they could visit the rest of the site. Even when they reached the form later, they complained about the form’s length and content.

Implication: Participants often considered leaving the site if they encountered the form early on, because, at that point, they had no evidence of the site’s value to them. When they encountered the form later, they were not likely to want to fill it out due to uncertainties about the benefits they would receive and concerns about whether they would be “spammed.”



Recommendations: Try to make sure the Advisor Service is not the first place site users visit. Whenever they visit the Advisor Service, make the first screen easy. Consider allowing visitors to simply enter their contact information and comments/questions (free form text). More detailed user information can be requested, as needed, in subsequent rounds of communication. State prominently, at the top of the form, how the visitor will benefit from filling out the form, and include a prominent privacy statement.

Improve Field Studies

The field studies were generally liked, although participants desired more detail about techniques used, benefits, before-versus-after comparisons, costs, timelines, personnel requirements, contractors used, etc. Participants were also unclear about what role, if any, betterbricks.com played in the field studies. The use of the 360º-view option and the inability to resize the field study window were operational difficulties that limited usefulness.

Key Finding: While participants found some of the field studies through the use of examples, they rarely found the full list of available field studies on their own. This seems to be due to field studies being listed under a submenu of the tabs at the top of the page.

Implication: With the current design, participants are likely to miss a much-valued and persuasive aspect of the website.

Recommendation: Make it more obvious that there is a collection of field studies and clarify how to reach them.

Key Finding: Eight of seventeen participants did not notice the viewpoint options at the top of the field studies because they are small and not highlighted. Therefore, many did not explore the 360º-view option. However, once noticed and used, participants valued these mechanisms.

Implication: These “value added” mechanisms will not routinely be used under the current website design, decreasing the vitality of the site.

Recommendation: Look for ways to make these options more prominent by increasing their size or mentioning their existence at the bottom of the text for each field study.



Key Finding: Participants who found the 360º-view generally did not know how to operate it. Most thought it would show a movie or a single, different viewpoint. When neither happened, they tended to click on the “download” link without reading the instructions. Another problem with the 360º-view tool is that there is no apparent way to return to the previous state. In addition, the field study window cannot be resized.

Implication: Once again, participants will not benefit from a valuable enhancement to the website.

Recommendation: Make the 360º-view tool easier to use by simplifying the instructions (e.g., “Please put the cursor over the picture and click to move viewpoint.”). In addition, develop an easy way to return from the 360º-view tool and allow for resizing of the window.

Response to the Website Usability Test Recommendations

The results of the Usability Testing of V1.0 of betterbricks.com were presented to the EBPI Steering Committee September 20, 2000. The Steering Committee requested a proposal from Cole & Weber to address the recommendations for general site usability improvement. Issues that Cole & Weber were not asked to address included expanded content and improved field studies.

The proposal was approved and website modifications were made and tested with four focus groups. Version V2.0 of betterbricks.com was launched in late January 2001. Recommendations addressed by the revision include:

Ø Homepage Redesign: Changes include the visual element on the homepage, reordering of the topics Discover/Believe/Act, moving the site tour from the homepage to the sitemap, providing access to information about the betterbricks.com sponsor, and changing the animation toward the center panel.

Ø Field Studies Pop-Up Window Redesign: Revisions include changing the color scheme, changing the 360º-view to a "virtual tour," and changing the guide to field studies.

Ø Improved Navigation Functions: Changes include navigation renames, improving functionality of the drop-down menus, increasing font sizes, and including visual elements to indicate functionality.



Ø Improved Messaging and Content Expectations: Improvements clearly indicate the sponsor’s non-profit status, include a link to the Alliance on the About betterbricks.com page, limit access to the Advisor Form to the Act section, change the description of the advisors, and reduce the number of fields on the Advisor Form.

WEBSITE USAGE

Introduction

The website and the Advisor Service found through accessing the website are the primary products of the betterbricks.com branding campaign. In order for people to access the advisory desk they must first access and use the website.

In this section of the MPER we examine data collected during website visits, analyze the relationship between advertising reach and website hits, and review the survey results from repeat visitors to the betterbricks.com site.

Advertising Reach and Website Hits

We wanted to understand the relationship between advertisements for betterbricks.com and visits to the betterbricks.com website. We looked at weekly data on the number of unique people visiting the website, the number of viewers estimated to have seen betterbricks.com TV ads, and the number of people subscribing to magazines in which betterbricks.com advertisements ran.8 Table 26 displays the totals and maximums for these characteristics.

We first normalized the three data sets because the variables were of such different orders of magnitude: 1,262 for the maximum number of unique website visits per week; 1,481,548 for the maximum number of print ad readers; and 12,552,000 for the maximum number of TV ad viewers. By dividing each observation by the variable’s maximum, each normalized variable had a value ranging from 0 to 1.

8 A unique visitor is a count of a visit from a URL. The visitor may view multiple pages on the site, or there may be

multiple visitors from the same URL. In each case the URL would be counted one time.

TV advertising viewers used "gross impressions.” Gross impressions are the total number of times a media campaign is viewed. Impressions come from multiplying the individual program audience (in thousands) by the number of times the spot ran.

Print ad viewing counts sum the circulation rates for each publication for each time period the advertisement is placed.



Table 26: Characteristics of Website Hits and Advertisement Reach

CHARACTERISTIC UNIQUE WEB HITS TV GROSS IMPRESSIONS

PRINT AD CIRCULATION

Totals May 2000 - March 2001 15,905 80,092,188 80,108,093

Weekly Mean 331 1,668,587 26,264

Weekly Maximum 1,262 12,552,000 101,523

Weekly Minimum 89 0 0

We also had to estimate the total TV viewing audience for September in the Pacific Northwest. At the time of this analysis, we only had Seattle and Portland estimates. Based on TV viewing data for the six weeks in May and June, however, we determined that Portland and Seattle viewers comprised 64% of the total viewers. Using the inverse of 64%, we then were able to estimate the total TV viewing population for September.

Our statistical analysis of the data shows that unique visits to the betterbricks.com website by individuals were highly correlated with the showing of TV ads (two-tailed Pearson r = .88, significant at the .01 level). On the other hand, unique visits to the website were not correlated with the number of people exposed to print ads (r = -.06, not significant at the .05 level).

As shown in Figure 4, below, the visual profile of unique website visits closely resembles the profile of exposure to TV ads, which aired in May, June, and September 2000, while the same is not true for the print ads, though a small "blip" can be observed in web hits following placement of ads in regional Sunday papers in February 2001.



Figure 4: Website Hits, TV Ads, and Print Ads

Web Hits to Advertising Campaign

0

0.2

0.4

0.6

0.8

1

1.2

5/1/00 6/1/00 7/1/00 8/1/00 9/1/00 10/1/00 11/1/00 12/1/00 1/1/01 2/1/01 3/1/01

Date

No

rmal

ized

Dat

a

WebHits TV_Ads PrintAds

Website Hits and Target Market Size

Another question about website visitors is whether the number of unique visits identified at the website tracking system (~16,000) are consistent with the number of site visits estimated in the follow-up survey (1% of the target market). The population of the Pacific Northwest between 18 and 65, according to the 2000 Census, is just over 7 million.9 One percent of 7 million is 70,000. This would lead one to believe that a total number of hits of 17,000 is quite low. Similarly, the total estimated gross impressions for the television advertisements were 12,552,000, again suggesting that 17,000 web hits is quite low.

However, the appropriate base for assessing this is the size of the target market, not the gross impressions or the adult population of the region. To assess the reasonableness of the estimate derived from the follow-up survey (1% visiting the betterbricks.com website), we developed a "back of the envelope" estimate of the

9 2000 Census results released by American Fact Finder: http://www.census.gov/clo/www/redistricting.html. The

early release provides counts for over-18 and for total population; we adjusted these to subtract over-65, based on the 1990 census.



target market. This estimate is provided in Table 27 and suggests that the target market as defined for the RPIP betterbricks.com campaign is about 130,000.

Table 27: Estimate of Target Market for RPIP Campaign

CATEGORY ESTIMATED TARGET MARKET A

ESTIMATED TARGET MARKET B

1999 Non-Farm Employees A 4,146,107 4,146,107

Employees in Service Firms and Government = 72%B 2,985,197 2,985,197

Firms > 20 Employees = 70%C 2,089,638 2,089,638

All Employees 2,089,638

Influential/Decision-Makers = 7of 100 Employees D 146,275

Access to Internet at Work and Home = 80%E 1,671,710

Influential/Decision Access to Internet at Work or Home = 90%D 131,647

Influential/Decision Makers Visit betterbricks.com=1%D 16,717 1,316

a. Oregon = 1,210,750; Idaho, 423,615; Montana, 277,144; Washington, 2,134,598 = total 4,146,107 1999 Non-farm employees in the Pacific Northwest, U.S. Census Bureau.

b. Bureau of Labor Statistics Employment Projections.

c. Analysis of 1997 Economic Census for RPIP targeted sectors.

d. Derived from RPIP follow-up survey.

e. Estimate for commercial employees, Question was asked differently in RPIP Baseline survey.

Table 28 displays possible estimates of the target market under two different assumptions. Target Market A is defined as the market initially identified for the RPIP campaign: all commercial employees in offices, education, and retail and government with over 20 employees. Target Market B is the target market the evaluation has tended to find as most interested in the campaign: influential staff and decision-makers at offices, education, retail, and government with over 20 employees. It is apparent that if either Target Market A or B are assumed, the number of unique website hits is satisfactory relative to the size of the target market.



Return Visitors Survey

When visitors return to the betterbricks.com website, they are asked to complete a short six-question survey. The purpose of the survey is to provide a rough assessment of the characteristics of website visitors. This section discusses the results of the survey for the period from May 1 to March 30, 2001. The survey is easy to complete and easy to skip. It appears as a pop-up window with a simple request to take a minute to complete it.

During the eleven months of operation, 556 visitors completed the return visitor survey. (Note: It is not possible to know the percent of unique hits or if the same visitor filled the survey out more than once.) Of the 556 visitors, 402 (72%) were from the Alliance target area, with most from Washington and Oregon, as shown in Table 28 below.

Table 28: Website Return Visitors by State

STATE COUNT PERCENT

Idaho 41 7%

Montana 15 3%

Oregon 130 23%

Washington 216 39%

Other USA 39 7%

Outside USA 115 21%

Total 556 100%

Most of the return visitors work in companies with less than 99 employees. Table 29 displays these characteristics.

Most of the return visitors come represent the private-sector – privately-owned firms (54%) or publicly owned (22%) – with 11% coming from government, and 9% from not-for-profit organizations. Four percent did not indicate organization type.

Finally, return visitors were asked which of the following four areas were of interest to them: financial outcomes, business process, stakeholder and customer relations, or human resources. (Note: these categories are not explained in the survey.) Five



hundred thirty-four visitors expressed interest in at least one of these four, with many expressing interest in more than one.

Table 29: Return Visitor Job Position by Firm Size

POSITION 0-99 100-499 500-999 1000+ TOTAL

Employee 196 37% 79 15% 27 5% 80 15% 382 71%

Employer 82 15% 16 3% 4 1% 8 1% 110 21%

Other 33 6% 7 1% 0 0 2 1% 42 8%

Total 311 58% 102 19% 31 6% 90 17% 534 100%

Table 30 shows how many visitors were interested in each area by job position. These data suggest that Stakeholder/Customer Relations generated the least interest across the three job categories, while Business Process generated the most.

Table 30: Interest Area by Job Position of Return Visitor

INTEREST AREA EMPLOYEE EMPLOYER OTHER TOTAL (N=551)

Financial Outcomes 209 25% 74 26% 23 27% 306 55%

Business Process 243 29% 83 29% 25 29% 351 64%

Stakeholder/Customer Relations 169 20% 58 20% 17 20% 244 44%

Human Resources 225 26% 72 25% 20 24% 317 57%

Total Statements 846 287 85

Percentages total more than 100% due to multiple responses.

BETTERBRICKS.COM ADVISORS’ EXPERIENCE

Construct, Inc. manages the Advisor Service of the betterbricks.com website. Website visitors who wish advisor services can submit a request (via e-mail or other means) for professional assistance. Anyone who hears of these services through



sources other than the website can also submit requests by a phone call to the betterbricks.com Advisor Service. Construct, Inc. screens inquiries, and for those that qualify and are interested, offers the services of a technical advisor who can conduct a free, four-hour, on-site assessment of the visitor’s commercial project. 10

To assess the use of the Advisor Service, we reviewed the August 16th Construct, Inc. report to the Alliance; we also received updated data for August 16th to March 1, 2001. 11 Between May 1, 2000, and March 1, 2001, 151 inquiries were made, all but 36 of these entries are dated, and Table 31 displays the distribution of contacts by time period. While most of the contacts occurred during the two betterbricks.com television advertising campaigns, the period since the cessation of the television ads has resulted in about the same number of requests for advisor services as during each of the television advertising periods.

Table 31: Timing of Requests for Advisor Services

TIME PERIOD PACIFIC NORTHWEST

OTHER USA INTER-NATIONAL

TOTAL N

Prior to July 11, 2000 39 4 2 45

July 12-September 21, 2000 6 6

September 22-October 14, 2000 20 8 3 31

October 15-March 1, 2001 14 16 3 33

Total 79 24 6 115

The 151 inquiries can be viewed in a variety of other ways, based on the contact’s location, job and firm type, and project characteristics. More data are available for the 139 who filled out the original form in the Advisor Service. The revised short

10 Qualified projects are remodel, retrofit, renovation, or new construction projects that can benefit from the

services offered by an advisor. There are no size or personnel category requirements, only that it be a real construction project. Some advisor requests can best be served by suggesting that the requestor use other Alliance projects such as the Seattle Lighting Design Lab or Energy Ideas Clearinghouse, or perhaps that they use their local utility or state energy office. The purpose of the form and the contact with Construct, Inc. is to ensure that the right resources are matched to the requestors needs. Some advisor requests have no need for a technical response and are addressed directly by Construct, Inc.

11 Cole, Jeff. (2000) Betterbricks.com Help Desk and Advisor Activity Summary Report . Presented to the Steering Committee August 16, 2000.



form, completed by 11, asks fewer questions and therefore contains much less data for analysis. Of the 113 contacts that gave their address, 78 (69%) were from the Alliance target area, as shown in Table 32 below.

Table 32: Advisor Contacts By Location

LOCATION TOTAL N PERCENT

Idaho 14 12%

Montana 5 4%

Oregon 22 19%

Washington 37 33%

Non PNW 27 24%

International 8 7%

Total 113 99%

Total percent may not equal 100 due to rounding.

One hundred nine of the contacts provided their job category and 88 provided information on their firm type.12 Tables 33 and 34 show that the contacts fall under the audience and firm types that the betterbricks.com campaign targets.13

12 Because of the complex way in which data were collected on job category, it was not possible to provide a

distribution by location.

13 Business decision-makers were the owner or general manager; influential employees were managers, senior managers, human resources, etc. Supply-side included architects, consultants, engineers, developers, and designers.



Table 33: Advisor Contacts By Job Category

JOB CATEGORY TOTAL N PERCENT

Business Decision Maker 35 32%

Influential Commercial Employee 23 21%

General Commercial Employee 30 28%

Supply-side 21 19%

Total 109 100%

Table 34: Advisor Contacts By Type of Firm

TYPE OF FIRM PACIFIC NORTHWEST

OTHER USA INTER-NATIONAL

TOTAL N PERCENT

Commercial/Office 36 16 4 56 64%

Educational 6 6 7%

Government 7 1 8 8%

Hospitality 1 1 2 2%

Industrial 3 1 4 5%

Medical/HealthCare 6 6 7%

Retail 4 2 6 7%

Total 63 20 5 88 100%

Total % may not sum to 100 due to rounding

There were 78 requests from firms in the Pacific Northwest who provided information on the long form.14 On the long form, contacts were asked to indicate all of their reasons for contacting the advisors. Table 35 displays the reasons given.

14 The reader may recall that the V1.0 website usability test found the advisor form to be a barrier. V2.0 of the

website includes a shortened form.



Table 35: Reasons for Contacting The Advisor Service

REASON NUMBER (N = 78*)

Interested in Information to Influence Decision-Makers 13

I Am a Decision Maker 39

Information on Technologies 37

Information on Design 26

Research on Improved Productivity 33

Help to Diagnose Facility Problem 20

Help to Define Project Scope 12

Help to Organize a Team 6

Help to Manage a Building or Remodel Project 17

Total Number of Responses for 78 Respondents 203

* Multiple Responses Permitted)

As can be seen in Table 35, there were 55 requests for help with commercial project opportunities. After further screening, eight of those requesting help were connected to a technical advisor for the on-site assessment.

Three of the 55 projects came to the Advisor Service after the short form was introduced. Therefore, the following tables only address the first 52 projects. As shown in Tables 36 and 37, the 52 projects that received further screening were primarily under 50,000 square feet, and most involved moves to a new location or new construction.

Further analysis of the data suggests that while the demand-side groups targeted by the betterbricks.com campaign are contacting the Advisor Service, their ability to bring viable projects (those that qualify for the on-site technical assistance) has been more limited than supply-side contacts.



Table 36: Size of Projects Receiving Further Screening

SQUARE FOOTAGE NUMBER PERCENT

Under 10,000 Square Feet 21 40%

10,000 to 50,000 Square Feet 20 39%

50,000 to 100,000 Square Feet 6 12%

100,000 to 500,000 Square Feet 2 4%

Over 500,000 Square Feet 3 6%

Total 52 100%

Total percent may not sum to 100 due to rounding.

Table 37: Type of Projects Receiving Further Screening

TYPE OF PROJECT NUMBER PERCENT

Move to New Location 19 36%

New Construction 13 25%

Renovation or Replacement of Building

9 17%

Routine Tenant Improvements 8 15%

Other 3 6%

Total 52 100%

Total percent may not sum to 100 due to rounding.

As shown in Table 38, of the eight projects that received help from a technical advisor, two projects came from demand-side clients (building owners, influential staff, etc.) via the website. The other five projects came from other sources associated with the project: Alliance staff or board members, Construct, Inc., or advisors themselves. Of the five projects from other sources, three were for business decision-makers or developers and two were for supply-side clients (architects or engineers).



Table 38: Client Category by Source of Contact and Total # Of Contacts

CLIENT PROFESSIONAL CATEGORY

NUMBER OF PROJECTS FROM WEBSITE

NUMBER OF PROJECTS FROM OTHER SOURCES

Business Decision-Maker 2 1

Developer 0 2

Architect or Engineer 0 3

Advisor Service Assessment

In March 2001, Construct, Inc. notified the Alliance that the contract for the Advisor Service was coming to an end and recommended that the contract not be renewed. Construct, Inc. believed that the Alliance could easily manage the level of traffic currently generated by the website for the Advisor Service. Agreements were in place with advisors and Construct felt that demand for the service would not outstrip the number of advisors in the near future. They felt that all the Alliance would need to do was to respond to requests for advisors, contact the requestor and clarify their need, and provide the requestor with information or access to an appropriate advisor.

The Alliance, however, determined that the Advisor Service was an important product that needed to continue while the entire commercial strategy was being revised. The Alliance asked Construct to continue managing the Advisor Service, at least in the short term, and a new scope of work is being negotiated. Construct anticipates the scope will include focused delivery of Advisor Services to architects, designers, and developers. They also plan to be able to respond “opportunistically” to prospects and have more flexibility to respond to specific needs of those requesting services.

Two major lessons learned emerged from the eight projects that were provided advisors and those two or three that came to the Advisor Service, but did not proceed to use an advisor. Most of the projects that used an advisor already had some connection to members of the Advisor Service or the Alliance. This meant that the client had confidence in the advisor and their ability to deliver a valuable service. The projects that did not proceed to use an advisor failed to do so because:

Ø The client did not have a connection to the service and thus did not have confidence in the service; and



Ø The client was proceeding through their design process so rapidly that the advisor could never actually get inserted into the process.

Several factors need to be addressed if traffic increases:

Ø The goals and objectives of the Advisor Service need to be well defined. Questions to be addressed include: Should the service only respond to website requests? Should it try to market itself? Is there an expectation in terms of numbers of projects, types of projects, etc.?

Ø The Advisor Service had no branded material to provide to clients, either in the form of Design Guidelines or even a brochure describing services and opportunities.

Ø Links to other organizations and Alliance projects are a good way to inform those who seek advisor services, but tend to detract from building the betterbricks.com brand.

Ø Credibility of the Advisor Service needs to be a component of the branding process. The Advisor Service had difficulty being seen as credible compared to local utility or local engineering/architecture firms.

SUMMARY OF WEBSITE RESEARCH

The goal of the website research was to determine the value of the website and assess how it can be improved. To do this, we conducted a usability test, reviewed hits to the website and compared them to advertising efforts, reviewed the return visitor’s survey, and examined use of the Advisor Service that was accessible through the website.

The primary goals of the website were to provide information to commercial sector employees, decision-makers, and influential staff, and to inspire them to seek to implement energy efficient solutions when they build, renovate, or remodel a commercial building. The website research suggests that the website did do this for some of those in the target market who encountered the website and were in the process of considering changes to their building space. The research also suggests that there are improvements that can be made to the site and to marketing of the site that are likely to make the site more effective.

The evaluation team conducted a usability test of the website in late August 2000, following the site’s launch in May 2000. The usability test found a substantial number of usability problems, many of which were addressed during a revision in



November 2000. A major lesson learned from this was that usability testing during site development can save valuable time and effort. Opportunities to improve the website that could not be addressed, given the resources available in November 2000, still remain.

The website received over 16,000 unique visits between May 1, 2000, and March 30, 2001. The timing of these unique visits is highly correlated (r=. 88) with the timing of the television advertisements. They were not correlated with the print ads (r=. 26). Although the number of unique hits appears to be small relative to the region’s population, when viewed in comparison to the size of the target market (130,000 to 1.6 million), and assuming 1% accessed the website in this first year, the website hits appear to be within a reasonable range.

Visitors to the website were asked to complete a survey if they returned. Five hundred fifty-six completed the survey, with three-quarters being from the Alliance target area. Most of the visitors were from private sector firms, but 20% were from government and non-profit organizations.

The Advisor Service was included as part of the betterbricks.com website. Website users could request assistance from the Advisor Service to provide information, answer questions or provide a four-hour, on-site assessment of their needs. The service received 151 requests for assistance. E-mail responses and links to other information resources satisfied most of these. Eight of the requests resulted in an on-site visit with an advisor; three of the eight came from business decision-makers, two from developers, and three from supply-side service providers. Further outcomes from these advisor services are not available at this time.

The findings from this research and from the follow-up surveys suggest that the website and the Advisor Service are primarily appealing to decision-makers, influential staff, and supply-side service providers.


CHAPTER 7: EVALUATION OF ENERGY CODE SUPPORT

INTRODUCTION

As discussed in the first chapter, maintaining and improving energy codes in the region is at the heart of EBPI, even if code support is not its most visible effort for a variety of political and market reasons. The Heschong Mahone report pointed out that the future of the energy codes infrastructure was not certain due to funding inconsistencies or declines. The report concluded that allowing that infrastructure to disappear was likely to result in fewer upgrades to the codes, the loss of experienced code people, and, ultimately, the deterioration of the energy codes themselves.

Thus, the Alliance, under EBPI, sent a Request for Proposals (RFP) to the relevant code agencies in Washington, Oregon, Idaho, and Montana in mid-1999. Washington, Oregon, and Idaho submitted proposals, and in the ensuing months contracts were let to fund code support in these states, with Washington receiving two grants. According to the EBPI project manager, a variety of attempts were made to involve Montana in the code support effort, but Montana chose not to request or receive funding. Contracts began in the fall of 1999.

EVALUATION PURPOSES AND METHODS

As one EBPI steering committee member put it, the reason to fund these code activities was simply to make sure “living, breathing, energy codes staff would remain on the job doing what they’re already doing.” Thus, the funding was targeted more at code maintenance than at code improvements. The Alliance also did not closely monitor the progress of the contracts since it was assumed that people would just keep competently doing their jobs. This evaluation, then, focuses on determining how well the funding served to maintain energy code infrastructure in the states that received funding.

To assess the code support projects we reviewed the RFP, the status of the projects with Alliance staff, and the DOE code website information for the three participating states. We also interviewed, by telephone, the key contacts on the four code-support contracts: one in Idaho, one in Oregon (with another staff member participating in the interview), and two in Washington (at two separate agencies). All respondents were very experienced with, and knowledgeable about, energy codes

Chapter 7: Codes Support Projects


in their state, and were very familiar with the activities the Alliance had helped fund. Each interview lasted about one-and-one-half hours and, while structured, tended to be quite open-ended. Thus, the basis of this evaluation is largely self-reports by those who received funding; however, we did ask respondents to tell us the reasons behind their assessments and to provide examples to corroborate their opinions.

We first asked respondents to assess if, and how well their funding had met the following goals for code support stated in the RFP:

Ø Help increase public support for, and awareness of, the benefits of energy codes to the environment, the regional economy, and building users;

Ø Facilitate the transition to a permanent, sustainable support structure;

Ø Help mainstream energy codes within the building codes;

Ø Encourage the support and participation of partner organizations; and

Ø Maintain strategic investments that maximize co-funding and participation by other institutions as well as private sector organizations.

In addition, we asked respondents about their:

Ø Specific code-related activities (specified in, or additional to the RFP);

Ø Relationship with the Alliance;

Ø Priorities for long-term energy code strategies; and

Ø Views of the future of energy codes in their state.

Their responses to each of these topic areas are analyzed in this chapter.

MEETING FUNDING GOALS

Overview of Goals and Activities

Table 39, below shows, by state, if the respondents thought they were trying to meet, through the grant, each of the Alliance’s overall funding goals listed above. Table 40 shows the types of specific work activities, as listed in the RFP, that recipients of funding might have carried out to meet the funding goals. (Please note that we have combined the answers of the two contacts in Washington.) Following



these two tables and the general discussion below, we will discuss each state’s specific situation in turn.

Overall, funding recipients felt they had been working toward the goals that the Alliance had set. When asked how effective they thought they had been in meeting those goals, most rated their efforts as very effective or between somewhat and very effective. All expressed a high commitment to their work. The only goal that was not uniformly pursued was that of seeking permanent funding (see Table 39).

Table 39: Was the Code Agency Trying to Meet These Alliance Goals?

IDAHO (N=1)

OREGON (N=1)

WASHINGTON (N=2)

ALLIANCE GOAL

YES NO YES NO YES NO

Help Increase Public Support for, and Awareness of, the Benefits of Energy Codes to the Environment, the Regional Economy, and Building Users

X X X

Facilitate the Transition to a Permanent, Sustainable Support Structure

X X X

Help Mainstream Energy Codes within the Building Codes

X X X

Encourage the Support and Participation of Partner Organizations

X X X

Maintain Strategic Investments which Maximize Co-Funding and Participation by Other Institutions as Well as Private Sector Organizations

X X X

As shown in Table 40, the work activities correspond well with those listed in the RFP, but also reflect the individual circumstances of the agencies. In Idaho, improving forms and inspections was not especially relevant when the push was to change the code. In Oregon, where the funding was the essential support for the day-to-day basic code activities, all activities were undertaken. And, in Washington, field inspections are not central to the services that either of the two code support agencies provides. These differences begin to hint at the quite different energy code situations (i.e., adoption, funding) in the three states, which will be explained further in the next section.



Table 40: Did the Alliance Fund These Types of Code Activities (as in the RFP)?

IDAHO (N=1)

OREGON (N=1)

WASHINGTON (N=2)

ALLIANCE GOAL

YES NO YES NO YES NO

Revise and Upgrade Energy Code Language X X X

Respond to Energy Code Inquiries from Building Departments and Applicants

X X X

Track Recurring Code Implementation/Interpretation Issues

X X X

Improve Energy Code Processing Materials and Forms X X X

Improve Field Inspection Tools X X X

Conduct Training and Education on Energy Codes X X X

Description of Goals and Activities by State

Idaho: $79,000 in Funding; September 1999 through June 2001

Key Accomplishment

In part, due to what the Idaho contract manager termed “essential Alliance funding,” the Idaho legislature has directed the State Building Code Advisory Board to make recommendations on statewide adoption of the International Building Code (IBC). The IBC embodies the International Energy Conservation Code and the International Residential Code, both containing energy efficiency standards for residential and commercial buildings. In mid-2002, the IBC should become the building code standard for the State, for all jurisdictions (cities and counties) that have previously adopted a uniform building code, and for any jurisdiction adopting a new code. Thus, energy codes are slated to leap forward in both stringency and consistency, since the majority of the population lives in covered areas.

Some threat, however, to these energy codes exists because the Building Code Advisory Board is considering recommending that Title 39 of the Idaho Building Code Advisory Act not include energy efficiency standards. Cities and Counties are currently making comment on the Act, and on June 18, 2001, the Building Code Advisory Board will take public testimony and make their decision. (In the past,



states adopting the UBC did not have to adopt the appendix containing the energy codes; but in the IBC, the energy codes are part of the general code.) Not surprisingly, this current state of uncertainty concerns support staff. There is also a more general concern about how energy code support will be affected by its move into the “mainstream” building codes department.

Further Background and Key Findings

Energy codes in Idaho are the purview of each local jurisdiction to adopt or not to adopt. If jurisdictions choose to adopt a residential energy code, it must be equal to, or more stringent than the statewide code adopted by the legislature (which is the Idaho Residential Energy Standard until the IBC, with possible energy code inclusion, goes into effect). While there is currently a non-residential standard throughout Idaho, actual energy codes have only been adopted by the state’s cities and counties.

Thus, energy codes have been inconsistently applied statewide or non-existent, although most of the larger jurisdictions do have at least a residential energy code. Code support has historically been housed in the Idaho Department of Water Resources Energy Division. On July 1, 2001, a Governor’s Executive Order requires the Energy Division to move two energy FTE’s to the Division of Building Safety, the state agency that handles other building codes.

When the IBC goes into effect in mid-2002, the energy codes that may be embedded in it would be the minimum standard. Idaho asked for the Alliance funding because they wanted to leverage Department of Energy money and their own state dollars to:

Ø Move toward consistent, statewide residential and non-residential energy codes; and

Ø Provide strong technical assistance to building code officials in cities and counties, as well as to builders throughout the state.

Like the other two states, Idaho code support activities do not receive state general funds. They have leveraged Alliance funds by using DOE funding in concert with Alliance dollars. The Idaho contract manager reports that the Alliance money has been “highly essential” in maintaining the current code situation; they have used the money sources together to provide more services and activities than they would have otherwise. In particular, they wanted to push hard toward code consistency and improvement by having the legislature adopt the International Uniform Building Code (IBC) as the statewide standard.



As shown in Table 39, Idaho tried to meet all but one of the Alliance’s goals – that of creating a permanent, sustainable code support structure. While Idaho code support staff would like to meet this goal, and have made efforts toward it, they report they need a certain level of ongoing support for training and there is no mechanism to supply that support. However, they have routinely sought and obtained other sources of money to pursue their work.

In terms of improving the energy codes, the funding was also essential and the results have been impressive: the legislature created a bill to establish a statewide IBC and it is scheduled to go before the 2001 legislature for confirmation and implementation in July of 2002. Thus, the state is moving toward a more consistent, more stringent set of energy codes, including the first non-residential code.

Otherwise, the contract manager reports that they have “very effectively” met all of the other Alliance goals. They point to using the Alliance funding to help support a very successful Idaho Energy Conference where they were able to network with public officials and others throughout the Northwest about code issues. They feel this conference was a key factor in increasing support among target audiences and furthering new code legislation. They also feel they have been effective in mainstreaming codes through working closely with Idaho Associations of Cities and Counties, developing a Handbook of Services that is well used, and partnering with the Alliance and other agencies and programs to support energy codes.

Oregon: $132,000 in Funding; September 1999 through June 2001

Key Accomplishment

Without Alliance funding, code support might not have survived in Oregon. Alliance dollars have allowed a myriad of code support activities to flourish in the state. For example, staff has been able to place downloadable code materials and forms on a website; this has made energy codes more accessible, made changing the code language easier, and made distribution easier and more cost-effective. Costs for changing/printing/distributing new forms have dropped from $9,000 to $1,000 per year. In addition, the funding has supported code maintenance and improvement efforts and has leveraged DOE money for special projects.


Oregon has developed mandatory statewide energy codes that establish a minimum that cannot be modified by local jurisdictions without state approval. The codes are



quite strong (they exceed 1995 MEC and ASHRAE/IESNA 9.0.1-1989). However, state general funds do not cover the cost of energy code support, and according to the code support staff, it is unlikely they ever will, as code funding is based upon building permit fees over which they have no control. Thus, code support and improvements are grant and contract driven, resulting in considerable uncertainty about the type and level of services that can be supplied. The Oregon Building Codes Division oversees adoption of any code changes. Code support functions are housed in the Oregon Office of Energy (OOE), the recipient of the Alliance funding.

OOE asked the Alliance for code support funding because they want to “capture energy resources through the code” and there was no BPA, DOE, or state money to cover their basic support activities (e.g., training, identifying upgrades, “handholding”). They believe, based upon their experience, that having a mandatory code is not sufficient for high compliance, especially because energy codes change and building officials change. In addition, all codes are subject to interpretation. They report they are often called on to be the “experts on how to apply the energy codes.”

In terms of code maintenance, the Alliance funding was “absolutely essential” in Oregon; there was no other source of funding. In terms of making code changes, the funding was also essential. OOE got involved with supporting various changes during the period of the contract, including resolving an issue with a new prescriptive fenestration path for the non-residential code.

Code support staff report they met four of the five goals for the code funding that the Alliance defined (transition to permanent funding being the exception), and performed all the code-related work activities listed in the Alliance RFP. They rated their efforts as quite effective in meeting Alliance goals. In terms of increasing public support and awareness of the benefits of energy codes, they report calls from architects and designers have increased due to activities funded by the Alliance – e.g., getting out and talking more with these groups and getting articles in code newsletters.

Authority over energy codes, as stated above, is housed with the Building Codes Division. Code support staff believe this mainstreaming, and their support of mainstreaming is crucial to the high level of code compliance in Oregon (i.e., the 100% energy code compliance reported in Ecotope’s 1994 baseline survey of residential buildings). Work activities funded by the Alliance – such as upgrading the energy code forms when changes occur, identifying how to make the forms easier to use, and streamlining inspection tools – contribute to meeting this goal.



Code support staff also credit Alliance funding with allowing them to get “most of the stuff” up on a website, including the code and the code compliance forms. With Alliance funding, code staff in Oregon can routinely partner with code officials, the Building Codes Division, architects, and homebuilders to provide energy code support and training. They also staff and provide technical assistance to the Building Codes Division Energy Committee.

Finally, OOE has leveraged Alliance dollars in order to obtain DOE funding for special projects (i.e., researching “server farm” energy saving opportunities). Notably, DOE funded the development of a slide presentation to compliment betterbricks.com that staff members have shown to user groups. The presentation cites the productivity and profit benefits of energy efficiency.

Washington: $70,000 for the State Building Code Council and $69,000 for the Washington State University Energy Program; September 1999 through June 2001

Key Accomplishment

Support from the Alliance facilitated more than 50 changes to the energy code in Washington. The code change process could not have occurred without this funding.


Like Oregon, Washington has developed its own fairly stringent statewide mandatory codes. The Alliance awarded code support contracts to two agencies in Washington that provide different types of support: the State Building Code Council (SBCC) and the Washington State University Energy Program (WSU). These two agencies perform different functions: the SBCC is a rule-making agency, established by the legislature to develop and adopt updated and amended versions of all the building codes; WSU provides advocacy and education specifically for the energy codes and was the entity that took over those aspects of code support when the Washington State Energy Office closed. WSU also provides technical support on energy codes to the SBCC.

In 1999, Washington was conducting a major review process of the residential and non-residential energy code, where over 100 changes to the code were being considered. The SBCC asked for funding from the Alliance, primarily due to declining funding for energy code oversight since 1995. Their goals in asking for funding were to be able to spend more effort talking with stakeholders about energy code issues, to identify problems with the code, and to develop and adopt new code language. WSU also was seeing their funding decline and wanted to be able to



continue being an advocate for energy codes, to conduct code training, and to contribute to code changes, especially in regard to improving the duct-sealing language and indoor air quality aspects.

In the case of the SBCC, the Alliance funding was essential to the code change process they were undertaking. The process resulted in many meaningful changes detailed in their report to the Alliance. Without the funding, the “process would have been on the shelf,” and only marginal changes would have occurred. DOE also contributed some short-term funding to support the same work.

In WSU’s case, Alliance funding was also very important. With that funding they could continue to provide basic services (i.e., answer questions from building officials, conduct training). It also allowed them to spend more time participating in the code change process and to serve on the SBCC Energy Code Technical Advisory Group. The funding leveraged DOE dollars as well ($1 of Alliance funding = $3 of DOE funding). WSU also received a small amount of funding from Puget Sound Energy.

Both agencies reported that they had met, or moved toward, all five goals set by the Alliance. As in Oregon, moving toward permanent funding was the most difficult target to achieve. Otherwise, both agencies felt they had generally been effective in fulfilling the goals. Although both agencies said they mostly worked with stakeholders rather than trying to increase broad public support for energy codes, SBCC said it publicizes its open meetings to a broader list, and also tries to reach groups who are especially affected by the codes (e.g., new homebuyers). WSU has a presence on the Internet and, like the SBCC, works with stakeholders.

The mainstreaming of energy codes seems to be a particular challenge due to the development of competing “national” model building codes by respected stakeholder organizations such as the International Code Council and the National Fire Protection Association. These model code groups promote the concept of including all building codes within an integrated set. The Washington State Energy Code is currently state-written. Adoption of a national set of codes would be a more “regular way of doing business,” by providing a national forum for consistency between states, and bringing a technical support, training, and certification infrastructure to the codes.

Both agencies felt the funding had been very effective in fostering participation of partner organizations and maximizing investments. The SBCC involved over 20 different members in its decision-making, and WSU is involved with a variety of groups from Master Builders to green building programs.



VIEWS OF ALLIANCE PROJECT MANAGEMENT

We asked respondents about a number of aspects of the Alliance’s project management for the code support projects. As shown in Table 41, the ratings for most aspects of project management were good for all three states. The areas where states gave lower ratings were project monitoring and communication with Alliance staff. Several respondents said they would have preferred a more “personal touch” and more frequent communication about how their work was progressing, as well as potential support from the Alliance. They also said, however, that they were quite self-sufficient at doing their jobs and did not want to be overly monitored. Finally, most states agreed that the Alliance’s payment procedures are excellent.

Table 41: Ratings of Alliance Project Management (n=4)

MANAGEMENT ASPECT EXCELLENT GOOD FAIR POOR DON’T KNOW

The Clarity of the Initial Request for Proposals

4

Contracting Procedures 4

Monitoring of the Project 2 2

Clarity about Deliverables and Time Schedules

4

Communications with the Alliance & Project Contact

1 1 2

Connection to Other Aspects of Energy Code Support in the Region

1 2 1

Payment Procedures 3 1

When asked what improvements could be made to project management, the most salient comments echoed the need for more frequent and regular communication, even though they all felt they had a good relationship with the Alliance. Several people noted that when the EBPI project manager left the Alliance in mid-2000, the attention to their contracts dropped to almost zero, until very recently.

In regard to monitoring, one respondent said he “wasn’t sure what the Alliance is monitoring.” He went on to say that codes are “esoteric” and technically



challenging, so that if the Alliance were really going to monitor the technical process, they would need to have a staff person to focus on codes. Another person pointed out that the Alliance might have missed opportunities to advance codes that they don’t even know about since communication and monitoring were so limited.

One fear respondents expressed is that the Alliance, by not being in more frequent contact, will not understand the complexities of the code environment, and will not be “in the loop” to foster code expertise and relationship-building among all the constituents of code-making.

FUTURE CODE OUTLOOK

Although an Alliance staff person is currently developing more in-depth information and recommendations for how the Alliance should continue to support codes in the region, we did ask respondents to look into the future of codes in their states and in the region. Table 42 shows how they rated a series of value statements that the Alliance had previously identified for guiding the course of energy codes.

As shown in the table, the respondents did have preferences and a fair amount of agreement about what were high and medium priorities. All three states agreed that the following elements are high priorities for a long-term strategy to maintain and improve energy codes in the Northwest:

Ø Energy codes should be simple to enforce.

Ø Building departments need sufficient resources to make energy codes work.

Ø Energy code support needs to receive permanent funding.

Most code support staff also felt that periodic code updates (to keep codes abreast of building technology) and continuing education for designers and building departments were key elements in a long-term strategy. Respondents said that balancing enforcing codes with design flexibility, recognizing that the private sector has an important interest and role in ensuring codes, and code consistency throughout the region were somewhat less important priorities. Finally, the respondents were split on whether or not energy codes should be mainstreamed with the rest of building codes.

We asked respondents several additional questions about the future. Their responses are summarized, by state, below.



Table 42: Importance of Elements in a Long-Term Energy Code Strategy

CODE ELEMENT HIGH PRIORITY

MEDIUM PRIORITY

LOW PRIORITY

Energy Codes Should Be Simple for Designers to Apply to their Buildings

2 2

Energy Codes Should Be Simple for Building Departments to Enforce 4

There Should Be a Good Balance between Design Flexibility and Enforceability

4

The Long-Term Need is to Keep Stringency Levels Abreast of Building Technology and Design Practices through Periodic Updates

3 1

Continuing Education, both for Designers and for Building Departments, is Important to Maintain and Improve Effectiveness of Energy Codes

3 1

Building Departments Need Sufficient Resources to Make Energy Codes Work

4

The Private Sector Has an Important Interest and Role in Assuring Effective Energy Codes

1 3

Energy Codes Should Be Mainstreamed with the Rest of Building Codes

2 2

Energy Codes Should Become More Consistent Between States and Regions, without Losing the Strengths of Existing State Codes

4

Energy Code Financial Support Needs to Receive Permanent Funding

4

Idaho

Idaho reports they are somewhat optimistic about the future of maintaining and improving energy codes in their state in what they see as a period of “substantial change.” On the one hand, they are very enthusiastic about the potential for a new, more stringent, state standard that has the ability to change building practices. However, as mentioned above, they are concerned about final legislative approval of the IBC, and continued funding support for energy codes in Idaho. They believe a good lobbying and stakeholder communication strategy will be key to legislative decision-making during the next session.



In addition, if the IBC goes into effect as planned, they realize a considerable amount of training and technical assistance will be needed with jurisdictions that are upgrading to the IBC or newly adopting the code.

They are in the process of helping the Idaho Association of Building Officials prepare a proposal to the Alliance to support such training and technical assistance. They say that without such funding, and without a strong energy code recommendation from the State Building Code Advisory Board, that the potential for implementation of the IBC in jurisdictions will be severely impaired.

When asked to provide advice for the Alliance regarding its code support activities, Idaho said that they are doing it on their own in large part, but they would like the Alliance to be open to ideas, to listen, and to be responsive. They would also very much like to obtain financial support to help insure energy code legislation and to implement the new code, including support to code staff who have moved to the Division of Building Safety. They feel this is a very critical period for energy codes.

Oregon

Like Idaho, Oregon is somewhat optimistic about the future of energy codes in their state. They say they have good relationships with code officials, provide services to them, and want to continue to do so. They say they are entering a period of “moderate change” over the next few years, especially with the potential for more power crises and, on the plus side, the development of new energy saving technologies.

Oregon code staff feel that the most important issues facing them are the limited electrical resource – especially peak demand – issues that they have not really had to face before. Another very important issue is how to move along and incorporate new technologies such as the use of steel framing, new ventilating techniques, and increased use of daylighting. All these factors contribute to a realization they are likely to need to upgrade training and compliance efforts. That, in turn, leads them to be concerned about the stability of their funding.

They are very hopeful that they will be able to continue to receive funding from the Alliance for basic education and other types of energy code support, since the state budget is extremely constrained. They feel energy code support activities would be dropped in Oregon without outside funding.

Oregon’s advice to the Alliance is to remember that energy codes need ongoing support in terms of technical assistance, education, and code upgrades. They feel these functions are very unlikely to be privatized in Oregon. Even though it may



seem that compliance “should be happening anyway” because the code is “law,” their experience shows that this is not the case. Without continued code support, they feel code compliance will decline.

Washington

Washington appears to be the most optimistic among the three states about maintaining and enforcing energy codes in the next few years. They say that with energy shortages and rising costs, the need for codes is reinforced among both politicians and the general public. They anticipate a period of moderate to substantial change, due in part to what one respondent termed the “whole model codes fiasco” (described below) and the way codes are developed.

Major issues facing energy codes in Washington include how to deal with the various versions of model building codes (that include energy codes) proposed by multiple code organizations, and the turf wars that have developed. This includes how to integrate current codes with the International Building Code. In addition, they want to keep upgrading the codes (i.e., a more stringent envelope for the residential codes, especially windows).

The State Building Code Council thinks it is critical to have permanent state funding so that energy codes are treated consistently from year to year. While they might ask for more funding from the Alliance, their goal is to get state funding. It appears they might be willing to let Alliance funding go by, and code development decline, in order to convey the need for permanent funding. They would, however, appreciate lobbying help from the Alliance in convincing the Washington legislature to provide permanent funding.

The Washington State University Energy Program, on the other hand, feels it is critical to have ongoing funding from the Alliance to satisfy the 25% match requirement for their DOE monies. Without this support, they are not sure they could continue their services. They will look elsewhere for funding, if not through the Alliance, but feel that the amount of change facing codes in Washington makes it imperative to keep up Alliance support.

The code staff in Washington offered the following advice to the Alliance about their future role in supporting and revising codes in the region. First, they would like to see the Alliance be more involved and committed to ongoing code enforcement and development – to be “players” on a regular basis. They feel the Alliance could play a critical role with codes stakeholders (i.e., building code officials, elected officials, fire officials), particularly by helping them move through the universe of technical and other issues energy codes face.



Finally, they would like the Alliance to bear in mind that energy codes have proven time and again to save more energy, more cost-effectively, than any other activities. They pointed out that codes are the last step in market transformation and that getting new technologies and other upgrades into code is the “ultimate exit strategy.”

CONCLUSIONS

We make the following conclusions as to energy code support:

1. As intended, Alliance code support has worked effectively to maintain codes. Without the funding in Idaho, Washington, and Oregon, energy code support and maintenance would have been, at the least, impaired and, at the most, non-existent. Over time, the lack of code support would probably result in lower code compliance (i.e., through less well-trained code officials) and less code development.

2. If it is accurate that the Alliance primarily funded code support just to be sure that code staff with institutional memory were kept at their jobs, the results show the states leveraged the money well beyond its intent. In Idaho, it helped with the preliminary adoption of the IBC; in Oregon, it sponsored innovative solutions, such as web access to materials and forms; and in Washington it contributed to 50 to 60 specific code improvements (although a few key changes did not occur).

3. The investment in codes appears to have been a very cost-effective way to spend EBPI funds; it appears the codes are not only being maintained, but improved as well.

4. Code staff in each state appear to be very conscientious in their duties and well connected to other relevant entities in their state and, through DOE, in the region.

5. Given the effectiveness of the dollars spent, funding should be made available to all four states, if they can justify the need. Principally, ongoing support will likely be needed in Oregon, Idaho, and Washington (at least for WSU), although the type of funding needed and the agencies receiving the funding will probably change somewhat over time.



6. The Alliance could benefit greatly by having a stronger presence and involvement in code issues and policy throughout the region. This is especially important in terms of supporting code changes that the states are trying to achieve and in helping to resolve code “turf” wars. While DOE has fostered regional coordination among the code agencies with its regional code collaborative, these same agencies would like to see the Alliance provide more technical expertise and higher-level policy and political support. In Washington, for instance, some code language improvements for the non-residential sector did not pass, and, according to several sources, Alliance support was missing and would have been helpful.

7. If the Alliance chooses to continue funding state codes program staff, communications need to be enhanced between the Alliance and its contractors. The greater Alliance presence would not only keep them in better touch with code progress or threats, it would also be a credit to the organization. The Alliance is well respected by code agencies in the region and could increase its influence, but it would need to make a strong commitment, given the complexity of the codes and the differences among the states.


CHAPTER 8: STATE SPECIAL PROJECTS

INTRODUCTION AND METHODS

Under EBPI, State Special Projects solicits projects that intend to meet one or more of these three overall goals:

Ø To generate innovative ideas to support energy codes

Ø To encourage voluntary actions that go beyond current code

Ø To foster partnerships in the building industry

In all, about $430,000 has been devoted to State Special Projects under EBPI. In the next sections we describe project goals, key findings, and accomplishments, followed by conclusions. The findings and accomplishments are based largely on telephone interviews with contractors for the eight projects that the Alliance funded. We also reviewed the contracts, progress reports, and larger reports for these projects. Only three of the eight projects are complete. We have based the discussion in this chapter on the following projects:

Ø Green City Buildings – Applying the LEED Rating System: The City of Portland Energy Office (now the Office of Sustainable Development) developed three demonstrations applying the LEED “green building” rating system to existing buildings. Its intent was to investigate whether green buildings are cost-effective.

Ø Demand Controlled Ventilation – DCV: The Oregon Office of Energy (OOE) is researching the efficacy of using CO2-sensors for controlled ventilation in assembly spaces and, if feasible, developing code language specifying its use.

Ø Idaho Energy Code Support Project: The Idaho Association of Building Officials (formerly the Idaho chapter of the International Congress of Building Officials or ICBO) is partnering with small cities and counties in southern Idaho to help them adopt local codes, set up building departments, and train staff.

Chapter 8: State Special Projects


Ø Gemstar: This Idaho Department of Water Resources (IDWR) project is designed to market the Home Energy Rating System (HERS) to residential builders and consumers

Ø Special Plans Examiner/Inspector Certification Project (SPE/I): This project helped maintain the private sector SPE/I Program in Washington. The funding helped support a round of exams, a database for referrals to certified examiners and inspectors, and transfer of the program to a Washington code agency.

Ø Oregon Code Website: The Oregon Building Officials Association (OBOA) is placing Oregon energy code interpretations and code compliance documents on a website.

Ø Shorebank Pacific: Shorebank Pacific provided training to help launch the Green Building Rehab Program, a program through which low-income homeowners can obtain loans to improve the energy efficiency and overall health of their homes.

Ø Prescriptive Residential Duct Code: The Oregon Office of Energy has developed prescriptive code standards to improve duct tightness in residential forced air systems. The code changes are currently under review by the Building Codes Structures Board.

MEETING ALLIANCE GOALS

Meeting State Special Projects Goals

The respondents were asked to rate themselves on whether their project met each of the three State Special Projects goals, and, if so, how. It should be noted that while the Alliance had these goals in mind, contractors who were awarded funding may or may not have been aware of these goals. In addition, it was not necessary to meet all the goals in order to receive funding, and some accomplishments may serve more than one goal. While the respondents have a vested interest in meeting these goals (and making their projects look valuable), we feel their ratings tend to hold up very well when the intent and accomplishments of each project are considered independent from the self-ratings. Findings by project are summarized in Table 43 below.



Table 43: Did the State Special Projects Help Meet These EBPI Goals (n=8)

EBPI GOAL GREEN BUILD-INGS

DCV IDAHO CODE

SUPPORT

GEM STAR

SPE/I OR CODE

WEBSITE

SHORE-BANK

PACIFIC

RESIDEN-TIAL

DUCT CODE

Generate Innovative Ideas to Support Energy Codes

Yes Yes Yes Yes Yes Yes No Yes

Encourage Voluntary Actions that Go Beyond Current Code

Yes Yes No Yes No No Yes No

Foster Partnerships in the Building Industry

Yes Yes Yes Yes Yes Yes Yes Yes

Innovative Code Support

All but one of the contractors indicated their projects brought innovative code support ideas to the table, as illustrated in the key accomplishments below:

Ø The Demand Control Ventilation technology uses CO2 sensors to detect how many people are in a space and then to adjust ventilation systems accordingly. The intent is to have this technology adopted into code through proving its technical viability and gathering buy-in from affected parties. The project is using a national set of experts to advise it; at the same time, this builds support from key stakeholders. (Alternatively, it could become voluntary standard practice, which would meet the second goal of increasing voluntary efficiency beyond code levels.)

Ø SPE/I borrowed the idea of using third-party examiners/inspectors from building departments that routinely use outside parties for special inspection situations such as welding. This created a structured system familiar to jurisdictions, yet provided the expertise needed for ensuring higher compliance with energy codes. At the same time it lightened inside inspection loads.



Ø In a state where 50% of jurisdictions have no energy code, and where “international” and “environment” can be “dirty” words to many, the Idaho Code Support project has gotten small and rural jurisdictions to adopt the new energy code (part of the IBC). Listening to community needs, building buy-in among affected parties, having retired buildings officials train new building officials, paying new officials for training time, and providing technical support has so far brought 20,000 residents under an energy code where none existed before. The project set out to get two jurisdictions to adopt, and in the end they got five; they are still reducing their costs.

Encourage Voluntary Actions That Go Beyond Current Code

Some of these State Special Projects are specifically geared to support current code and were never intended to encourage “beyond code” activities (Idaho Code Support, OBOA Website, SPE/I). However, other projects are specifically targeted to voluntary energy savings. These projects show special promise with voluntary activities:

Ø Green Buildings supplied data that convinced Portland officials that green buildings are cost-effective investments. New city buildings and facilities are now required by City policy to be 20% more efficient than current energy code requires, and building retrofits must be 10% over code. (Thus, one might not consider this system “voluntary.”) In addition, due to Alliance funding, the City established a cross-disciplinary Office of Sustainable Development and provided a regular source of funding.

Ø Gemstar is marketing a residential rating system to builders and consumers that will encourage them to build beyond current code.

Foster Partnerships in the Building Industry

All of the special projects could point to relationships they were developing with the building industry, including these key accomplishments:

Ø Gemstar hires and trains private raters for the homes. They also partner with homebuilders and government agencies.

Ø Green Buildings did what is often nearly impossible: it brought city building project managers from different bureaus and built support for the



green building approach. As a case study, this can affect many more in the building industry.

Ø The DCV advisory committee includes designers, builders, owners, government, manufacturers, and others who come together via e-mail to discuss and resolve issues.

PROJECT DESCRIPTIONS AND KEY ACCOMPLISHMENTS

Green Building Baseline Study: $40,000 in Funding

Background and Goals

The City of Portland, Portland Energy Office, requested funding from the Alliance to support the City’s Green Building Initiative – an initiative designed to expand market demand for green buildings and to make green building practices easier to implement. During the course of developing and implementing this Initiative, interested parties raised two major questions: What constitutes green buildings? And, what are the costs and benefits of building green? Both of these questions signaled potential barriers to building and promoting green.

Thus, the objectives of the Green Building Baseline Study were to address these questions by:

Ø Determining how “green” three new City buildings were; and

Ø Assessing how each building could have been built “green” and the costs and benefits involved.

This study used the LEED system to rate the buildings, and considered direct, regional system, and societal costs. It found that the buildings did not meet LEED standards but that minimal investments in energy and environmental improvements would allow them to do so.

Key Accomplishments

The Green Buildings Baseline Study proved to be an essential catalyst in getting City agencies to cooperate in considering the efficacy of building “green” City buildings. Without the Alliance funding, the project contact said the study would not have happened.



The study was a mechanism to involve and invest various agencies, and, in the end, it produced hard, positive data about the cost-effectiveness of building green. This data allowed City officials to:

Ø Adopt a more stringent policy for energy efficiency in all new City buildings (with substantial buy-in from affected parties); and

Ø Spawn a new Office of Sustainable Development with regular City funding.

The study also can (and has) served as a case study for other agencies and entities to promote the cause of green buildings.

Demand Controlled Ventilation (DCV): $130,000 in Funding


The Oregon Office of Energy requested funding to determine the viability of a new technology – demand controlled ventilation or DCV – to save energy, and if viable, to have DCV routinely adopted by the building industry (either through new energy code language or simply because the technology works well and makes sense). DCV senses the amount of Co2 in rooms and adjusts ventilation according to the Co2 level, allowing air levels that are matched to the level of room occupancy. The process of investigating the viability of DCV involves the participation of a multi-disciplinary national panel of experts who communicate primarily via e-mail.

Key Accomplishments

The first phase of this project demonstrated that the DCV technology was worth further investigation, despite the fact that DCVs have had mixed reputations for performance over the years. The second phase of the project is still underway, but will provide a “best practices method” for the design community; design guidelines for specifications and schematics; control sequences of operations for engineers and facility managers; and recommended energy code language and code adoption strategies.

So far, the project manager points to the following indicators of success:

Ø The technical advisory committee remains actively involved in the process and their support will increase the credibility of the technology.



Ø Third parties have published their successful experiences with implementing DCVs and are referencing the project.

Ø Greater acceptance of the technology is appearing in trade journals.

Idaho Energy Code Support Project: $77,500 in Funding


The Idaho chapter of the International Congress of Building Officials (ICBO) initially requested a small amount of funding ($17,500) from the Alliance to provide training to Idaho building officials on a new, voluntary Idaho Commercial code. The requested subsequent and more substantial funding ($70,000) helped them partner with rural Idaho cities and counties to adopt local energy codes (based on the International Building Code), set up buildings departments, and train and equip staff. In Idaho, each jurisdiction must adopt its own energy code and half of them – mostly rural – have not adopted such codes. While this in no way equals 50% of the population, rural Idaho is an important component of the state’s make-up and politics.

Key Accomplishments

ICBO’s efforts have been both successful and exemplary in demonstrating how to bring energy codes to rural communities. They have gotten them adopted where no none have gone before, and 20,000 Idahoans now have energy codes. They have generated good will and this has influenced the legislature toward energy codes.

They have also learned from the failure of their first project, which appeared at first to have gone very well. The code was adopted, building officials were trained, the code was being enforced, and two other jurisdictions wanted to emulate Boise County. However, eight months later the country had a referendum and undid the adoption.

Rather than being defeated by this, ICBO rethought its strategy. They now spend more time involving the community in the process, making sure everyone is “under the tent,” and making sure everyone understand the benefits of adopting the code – especially the increased affordability of utility bills. They also have carefully worked out processes that make adoption credible (use of retired building code officials) and affordable (paying for training time of new code officials).



GEMSTAR: $60,000 in Funding


The purpose of this project, implemented by the Idaho Department of Water Resources, is to help promote GEMSTAR – a voluntary builder standard and home energy-rating program that IDWR developed. GEMSTAR uses a five-point rating system; a five-star home meets the 1991 Model Conservation Standard. Alliance funds will sponsor coop advertising for GEMSTAR builder and realtor partners, integrate efficiency information into key Idaho Multiple Listing services, and train realtors to use and promote GEMSTAR. GEMSTAR will be marketed to homebuyers, realtors, lenders, and builders. It will use private-sector partners from the building industry to do the ratings.

Key Accomplishments

At the time of the evaluation interview, the contractor reported that the next few months are critical. He had just recruited the raters and was about the train them. He reports he was able to recruit the raters because of the marketing capability promised by Alliance funding. The next steps will be to recruit partners for the coop advertising efforts.

Special Plans Examiner/Inspector Certification Project (SPE/I): $33,694 in Funding


The Alliance provided funding to the Northwest Energy Efficiency Council (NEEC) to make sure, in the short term, that the examination and certification process for Special Plans Examiners would continue, and that, if possible, the Washington Association of Building Officials (WABO) would continue to maintain a list of certified examiners. The Special Examiners may review plans for non-residential buildings and conduct field inspections to document compliance with the Washington State Energy Code. This allows building departments to outsource these tasks.

Key Accomplishments

The funding did ensure a new round of exams, the establishment of a database, and transference of SPE/I to a Washington code agency (not WABO). The third party



system is thus still in place to be used by building code jurisdictions. Prior research has shown SPE/I involvement upgrades code compliance and energy efficiency.

Oregon Energy Code Website: $58,000 in Funding


The purpose of this project is to automate the process for commercial and residential energy code compliance so that applicants are more likely to get the “right answer” for building envelope requirements, and so that costs to applicants and building departments for technical assistance are reduced. Once completed, the website will be marketed to potential users.

The original website plan called for an interactive, web-based wizard for building envelope requirements, an on-line searchable database of interpretations, and other sources of advice that would help builders demonstrate code compliance. The Alliance contracted with the Oregon Building Officials Association which, in turn, contracted with a website development/energy specialist.

Key Accomplishments

This project is only about 20% complete, with preliminary information gathered. The future of this project in uncertain.

Shorebank Pacific: $5,000 in Funding


Shorebank Pacific (a commercial bank) requested Alliance funding to help get the Green Building Rehab Program off the ground. Through the Green Building Rehab Program, Shorebank Pacific loans money to the Portland Development Commission (PDC) to help finance improvements in low-income homes so that the homes are healthier and more energy efficient. While this project did not specifically support energy codes, it definitely promoted voluntary building energy efficiency and partnerships in the building industry. The Green Building Rehab Program relies on partnerships among the bank, the PDC, the building contractors, materials providers, and the City of Portland (which performed the audits).



Key Accomplishments

The Alliance funds were used to sponsor training sessions in 1999 for PDC staff and for contractors that were going to do the remodels, when the program was just starting. While the bank was intent on supporting this program by providing money for the actual loans, the Alliance dollars paid for the training in full. According to Shorebank Pacific, they could not have done the training without the Alliance funding. The initial training was important to the program’s successful start-up, and the Green Building Rehab Program has been well received.

Shorebank Pacific is currently in the process of renewing the program with the PDC. According to the bank’s representative, Shorebank Pacific would like to pursue similar projects with the Alliance in the future. The bank is a “commercial bank with an environmental mission,” and they would be interested in providing financial support to other market transformation efforts.

Prescriptive Residential Duct Code: $25,000 in Funding


The Oregon Office of Energy (OOE) requested Alliance funding to develop code language to improve duct tightness in home heating systems, to modify the language through input from the building community, and to get the language adopted into code. While one overall goal was to get the language adopted as code, another goal was to educate code officials and the building community about best practices related to residential ducting. Once adopted, the proposed code changes will increase energy efficiency, improve health and safety, and offer prescriptive measures to implement a provision in the existing code that says that ductwork shall be air tight.

According to the OOE respondent, while the code language itself was not particularly innovative, it was innovative to build this project onto other existing and successful initiatives to help prove the technical and financial feasibility of these ducting changes. These existing initiatives included the Alliance-funded Performance Tested Comfort Systems project (which included a voluntary training and certification program), and the state tax credit for proper ducting. The Prescriptive Residential Duct Code project has promoted partnerships among building and heating contractors, code officials, and state code agencies (e.g., the Energy Committee of the Building Codes Structure Board).



Key Accomplishments

OOE had planned to propose duct code changes; however, the Alliance funding has allowed them to pursue this effort more thoroughly and to work with the Washington State University Energy Program to coordinate development of code language. The code development process included meetings with contractors and code officials to both educate them and get input to improve the proposed code changes. The code changes are currently pending at the Energy Committee of the Building Codes Structures Board.

While the proposed code changes are not yet adopted, the OOE respondent felt as though the project was “70% there” in terms of reaching the goal to educate code officials and the building community about the their benefits. However, more education is needed. The OOE staff person feels that he needs to get out and talk to more builders about the benefits of improved ducting, but it is difficult to get an audience with them. He believes that as more contractors understand the practices and their benefits, they will want to incorporate them voluntarily and they will support the code changes.

VIEWS OF ALLIANCE PROJECT MANAGEMENT

Table 44 shows how the eight respondents rated the Alliance on several project management attributes; these attributes are similar to those used in the Code Support Interviews in Chapter 7. Most ratings are at least “good” and a number are “excellent,” especially in terms of getting help from the Alliance and in contracting and payment procedures. In general, respondents were pleased that they were asked for feedback and hoped that there would be future opportunities to provide their perspectives.

Communication with the Alliance once again appears as the area where contractors think the most improvement could be made, even though overall ratings were positive. Several contractors mentioned they felt the Alliance did not think their projects were important, and one mentioned feeling like a “puppy out in the cold.”



Table 44: Ratings of Alliance Project Management (n=8)

MANAGEMENT ATTRIBUTE EXCELLENT GOOD FAIR POOR DON’T KNOW

The Clarity of the Initial Request for Proposals

2 3 3

Contracting Procedures 2 4 2

Monitoring/Reporting Requirements 2 5 1

Clarity About Deliverables and Time Schedules

2 5 1

Communications with the Alliance 3 3 2

Help from the Alliance When You Need It 4 2 1 1

Payment Procedures 5 1 2

ADVICE TO THE ALLIANCE

We asked each respondent the most important advice they would give the Alliance in supporting the three State Special Projects goals. Their advice often reflects the individual thrusts of their projects, but was thoughtful nonetheless.

Ø As a regional entity, the Alliance has a potential to bring people together that cuts across disciplines to deal with common problems. It appears the Alliance has many interesting things going on; greater communication would spur new ideas.

Ø The Alliance should take a more active role in the states’ code review and adoption processes, to really understand what is going on, and to make a positive contribution. The building code change process is a long-term process, which requires much vigil, attention, and education.

Ø If the Alliance is going to promote voluntary efficiency beyond code, they need to create products with clear energy efficiency “standards.”

Ø In local codes issues, no “one-size-fits-all.” Find out what jurisdictions are most concerned about and fit your product or service to those concerns.



Ø Support key cities in the region, even though the direct pay off may be energy savings and policy changes, not code changes.

Ø Do more to support codes. There are many problems and much to be done.

CONCLUSIONS

We offer the following conclusions regarding State Special Projects:

1. Overall, the State Special Projects that are complete or nearly complete can point to specific, valuable accomplishments that meet the goals the Alliance set for this type of project under EBPI. They appear to have been cost-effective investments.

2. Some of these projects, and their accomplishments, appear to be out of the loop in terms of attention from the Alliance. By paying closer attention and promoting these projects to other, similar organizations, the Alliance will get “more bang for its buck.”

3. Offering a flexible grant program like this has the advantage of assisting many different activities proposed by an array of public and private entities that may not fit into other larger, but more limited-focused programs (e.g., Code Support, RPIP). While their impact may not be felt region-wide, these smaller accomplishments in building technology and code development are crucial to “push the envelope” and offer specific, measurable case studies and solutions.




CHAPTER 9: SEED FUNDING AND THE NEW CONSTRUCTION BASELINE

INTRODUCTION AND METHODS

This chapter groups together evaluative findings for the Seed Funding Projects and the New Construction Baseline Survey. The Seed Funding category for EBPI, as stated in the introduction to this report, has always been a bit uncertain for the program. As far as we know, no Request for Proposals was ever let, and much of the original budget devoted to this area was used for other purposes. From reviewing old EBPI documents, the original goal of Seed Funding was to:

Ø Help fund private businesses that intend to be self-sustaining and that deliver services to help implement energy codes at the design and building levels. Through these businesses, the Alliance hopes to improve energy code compliance and upgrade building energy efficiency beyond minimum code requirements.

In this chapter we will briefly discuss the intent and results to date of the three major projects listed under Seed Funding as “Other Projects,” including:

Ø Future at Work

Ø Brewery Blocks

Ø New Buildings Institute

The New Construction Baseline Survey was designed to measure current energy efficiency practices in new residential and commercial buildings in the Alliance’s four-state region. This brief analysis explores the intent and outcomes from this aspect of EBPI.

The findings presented in this chapter are based upon interviews with the key contract staff and review of written materials generated for each project.

PROGRESS FOR FUTURE AT WORK

In the process of implementing the Advisor Service for betterbricks.com, Construct, Inc. became aware of a demonstration site in Seattle sponsored by Callison Architects, Sparling Engineering, Steelcase, and Turner Construction. In 2000, the

Chapter 9: Seed Funding and the New Construction Baseline


office demonstration site located in a Class-A office building in downtown Seattle provided a complete mock-up of an office environment for the year 2007. The office space incorporated advancements in communications, business standards, economic factors, real estate trends, and corporate sociology. More information about Future@Work can be found at its website.15

In late 2000, the Future@Work site was temporarily closed for redesign. The Alliance joined as a contributor and arranged for advisory services to be provided. According to Construct, Inc., the design team accepted many of the suggested changes for the space, including integrated lighting and mechanical changes. The Future@Work team wants to continue to include the Alliance in future activities, would like the Alliance to have an exhibit, and would like the Alliance to join as a full partner in the activity.

The redesigned site will reopen in May 2001, in time to be a major attraction for the International Development Research Council conference, which attracts corporate real estate executives. Anticipated key visitors to the site in the months and years after reopening include decision-makers from regional and national corporations, as well as developers and real estate executives.

PROGRESS FOR BREWERY BLOCKS PROJECT

The Brewery Blocks project evolved out of the need to have good case studies of energy efficient new commercial buildings for EBPI, and especially for the RPIP effort. The EBPI project manager thought the project had the makings of a good “poster child.” Brewery Blocks is a large, five-block, mixed-use commercial development in downtown Portland; it is preserving and renovating historic buildings and undertaking new construction. Construction on this project is just getting underway.

When the Brewery Blocks project came to the attention of the EBPI project manager, he saw its potential for a case study. The developers already had an interest in building “green” and they were open to having outside consultation on the environmental design components of the building. The EBPI project manager hired an initial consultant to work with the team; he conducted two “eco-charettes.” Among other things, these charettes helped define the mission, opportunities, and barriers for designing and building a resource and environmentally responsible building.

15 http://www.future-at-work.org/toc.html



When the initial consultant could not continue with the project, the EBPI project manager hired River City Resources to be the project’s environmental design consultant. Their job was to make sure that the energy efficiency potential of the project (and potential other environmental aspects as well) was realized. River City Resources worked with the integrated design team in this capacity during eight months of design development on the project – from April 2000 through the end of that year. The consultant attended weekly, integrated design team meetings, brought energy efficient design opportunities to the table, and helped the design team make decisions. The overall goal was to help the project meet the silver LEED rating (a measure of sustainability in buildings).

Key accomplishments of this project to date include:

Ø According to River City Resources, the architect and developer cited the environmental design role as critical to the sustainability aspects of project. They report that the building went further with energy and environmental investments than it would have without the consultant’s input (and EBPI/Alliance support).

Ø The project leveraged other dollars for the energy efficiency efforts through PGE’s Earth Smart program, paying for energy modeling the eco-charettes. The project has also applied for Oregon State Tax Credits. (However, Alliance dollars were critical.)

Ø The project, as designed, is slated to use almost 30% less energy than the current Oregon energy code. It will include innovative energy and other resource efficient elements, including a high efficiency HVAC system, a high efficiency district-chilled water plant, night flushing, high efficiency glazing, efficient lighting systems and controls, occupancy sensors in common areas, daylighting, operable windows, and more.

Ø The consultant and design team are developing a Tenant Manual that will help tenants maintain and enjoy the environmental excellence of the buildings. This will be the first manual of its kind, and it is hoped it will serve as a model for other environmentally-conscious commercial building projects.

Ø The developer paid for part of the costs of pursuing high levels of energy efficiency and has hired the current design consultant for further work, referring her to other projects as well.



NEW BUILDINGS INSTITUTE/DESIGN GUIDELINES

The New Buildings Institute (NBI) began working with the Alliance in February 2000. In the process of developing the RPIP betterbricks.com website concept, the project managers at Cole & Weber and the Alliance had realized that the betterbricks.com website would need collateral materials for the Advisor Service and for the Advocacy Program. The Alliance contacted NBI and they presented a strategy for developing collateral materials to the EBPI Steering Committee on February 10, 2000.

The Steering Committee agreed to the approach outlined in the presentation and allocated monies from Seed Funding to support the contract. A contract was finalized about two months later, though the work began immediately.

The initial plan for the effort was to develop Design Guidelines to address the topics covered at the betterbricks.com website: daylighting, thermal comfort, and electric lighting. There were to be two sets of guidelines: one for end-users in commercial buildings (demand-side) and one for the new commercial construction design community (supply-side). The guidelines were to be modeled after those developed by the Northeast Energy Efficiency Partnership Design-Lights Consortium and the Energy Center of Wisconsin Daylighting effort. The timeline was very short with the Advocacy Program scheduled to begin in late May and the Advisor Service to be activated concurrent with the May 1 launch of the website.

Several problems evolved during the design and delivery period. In April the project manager for Cole & Weber left Cole & Weber, resulting in some uncertainty as to the vision for the entire project and especially the guidelines.

After NBI developed its initial draft for the first guideline, comments on the draft suggested that NBI should develop one guideline that could meet the needs of both demand and supply-side readers. The comments on the draft also indicated that the content should be more technical and that graphics should be included. However, graphics were the responsibility of Cole & Weber, who refrained from working on graphics until the copy was finalized. With such uncertainty, the RPIP team decided to postpone implementation of the Advocacy Program to late July.

In August, the guideline vision was still unresolved and with the pending departure of the Alliance project manager in September, NBI and the Alliance agreed that the guidelines development work should be put on hold until the vision for the guidelines could be clarified.



In October 2000, the new Alliance project manager and NBI agreed on the direction for the guidelines: different guidelines for each audience (supply-side and demand-side) and three areas of focus. The guideline text for two versions of each of the three technical areas was completed in early 2001. Progress continues, with the daylighting guideline currently in draft final form.

A few of the lessons learned by the NBI in this process were:

Ø The guideline development process has underscored for NBI the need for even more versions. For example, each paragraph in a guideline targeted for a commercial business-decision maker probably requires a three to six-page technical guideline for the design professional and maybe a one-page guideline or cut-sheet for the contractor. Similarly, a guideline for a commercial employee can be much more general and descriptive of the aesthetic and emotional benefits, while one for a decision-maker needs to stress the financial return and effects on worker productivity.

Ø Another lesson is that it is difficult for technical energy people to translate technical issues into materials that will capture market attention. For instance, it can be difficult and time consuming to find the right graphics to convey complicated technical concepts. In addition, it is often hard to find the right balance between technical detail and making a guideline concise and readable.

Ø NBI also offered some insights about betterbricks.com overall. They feel that betterbricks.com needs to be managed like a product, not a program. A product needs leadership and a market manager, a visionary, someone whose sole function is to make the product a success in the marketplace. A program needs a manager who will make sure the program is delivered and implemented as intended. A product manager would look for strategic partnerships rather than contract work. A product manager needs to have long-term commitment and sufficient resources to survive any initial failure, revise the product, and keep going.

SEED FUNDING CONCLUSIONS

As with Code Support and State Special Projects, the intent, accomplishments, and benefits of the Seed Funding projects have somewhat fallen through the cracks of EBPI (at least according to the contractors involved).

The projects appear to be valuable in their own right and, at least in one case (Brewery Blocks), are spurring private businesses to be self-sustaining in providing



energy efficiency services, one aspect of the seed funding idea. However, the Seed Funding category under EBPI mostly appears to be unclear. The initial goals were not actualized and the Seed Funding monies were mainly redirected to projects the EBPI Steering Committee viewed as critical to the implementation of the RPIP.

It is impossible to say whether the initial goals could have been realized if they had been tried. However, the lesson from the Brewery Blocks suggests that there are business opportunities that the Alliance could stimulate with seed funding at the right time. The potential interest in such an approach could be explored if a RFP process were initiated.

NEW CONSTRUCTION BASELINE SURVEY

The EBPI Steering Committee contracted with Ecotope, Inc. to conduct a new construction baseline study across three market sectors: residential, multi-family, and non-residential. The study was funded in 1998, and conducted in 1999. All data had been collected and draft reports submitted by March 2000. As of May 2001, a draft final summary of the nonresidential baseline report has been prepared, with the residential summary to follow shortly.

The studies used stratified random sampling across size and building categories and included building, permit, and plans review, as well as interviews with relevant parties. The residential construction baseline was across the four states, while the multi-family construction baseline was just in Oregon and Washington. For the nonresidential conduction baseline, new building reviews were conducted for Idaho, Oregon, and Montana; while in Washington, a sample frame was developed but findings based on building review data from a 1996 study were used. Interviews, however, were conducted in all four states with key parties.

The studies essentially are structured to document the status of new construction practice across the four states. The goals of the studies were:

Ø Characterize progress in the new construction residential, nonresidential and multi-family markets.

Ø Assess construction relative to the specific code requirements in each of the four states.

Ø Assess attitudes to energy efficiency, barriers to energy efficiency, and attitudes to code decisions on the part of architects, engineers, owners, and developers across the four states.



Ø Assess market-specific levels of compliance relative to the Model Energy Code (MEC) ASHRAE 90.1 and the Oregon Energy Code.

Ø Develop technology baselines for lighting, HVAC, and envelope measures.




CHAPTER 10: EBPI PROGRESS INDICATORS

INTRODUCTION AND INDICATORS

The evolving nature of EBPI has meant that the Alliance has developed several versions of progress indicators for the initiative. In some cases, these indicators have not covered all aspects – for instance, they just focused on the RPIP. In all cases, the indicators have lacked specific guidance for evaluating whether efforts have been successful. For instance, if we are looking for changes in awareness due to RPIP efforts, how much change should we expect? How should we evaluate the finding that 4% of regional decision-makers and influential staff became aware of the betterbricks.com brand due to the RPIP?

For this evaluation, we have decided to use the nine short-term (contract period) outcomes listed in the EBPI logic model as progress indicators (see Chapter 2). These outcomes seem most appropriate to us because EBPI staff and the Steering Committee developed them once the EBPI design was fairly complete, although the Logic Model was never strongly endorsed by either. Still, given the other options, this set of outcomes seemed the best against which to try to measure EBPI’s performance.

Table 45 shows which EBPI elements were either primary or secondary links to each short-term outcome. Primary links to outcomes are in bold type, while secondary links are in Italics. We revised the nine outcomes with guidance from the Alliance to make them more straightforward (as shown in the third column of the table).

1. Demand for Professionals with Energy Efficiency Expertise Increases

The baseline level of demand for professionals with energy efficiency expertise has not been established among business influential staff, decision-makers, or the design community. Interviews with architects in the region suggest that demand may exceed supply, with several designers noting it is difficult to find energy efficiency experts in which they have confidence.

As shown by the Logic Model, EBPI had no elements that were directly linked to this outcome. The implementation of EBPI in 1999-2000 put little effort into creating demand for energy efficiency professionals. While Energy Code Support

Chapter 10: EBPI Progress Indicators


and State Special Projects did involve these professionals, they generally undertook infrastructure projects, not services to designers or business decision-makers.

Table 45: PROGRESS Indicators for EBPI

EBPI ELEMENT

(PRIMARY LINKS TO

OUTCOMES ARE IN BOLD; SECONDARY LINKS ARE IN

ITALICS)

CONTRACT PERIOD OUTCOME FROM LOGIC MODEL

REVISED OUTCOME

RPIP

Seed Funding

Incrementally improved marketplace (= demand) for professionals with energy efficiency expertise

Demand for professionals with energy efficiency expertise increases

RPIP

New Construction Baseline

Public (primarily nonresidential) awareness/curiosity of productivity opportunities in building and how they relate to energy efficiency is raised

Awareness of productivity and how it relates to energy efficiency increases among nonresidential demand-side actors

Seed Funding

RPIP

Advisory Group

Increased availability of energy efficiency professionals for folks to call

Availability of energy efficiency professionals increases

Seed Funding Decreased dependence on outside funding to support energy efficiency activities in buildings = moving toward self-sufficiency

Energy efficiency activities in buildings depend less on Alliance funding and move toward sustainability

RPIP

Seed Funding

Advisory Group

As a response to consumer requests, supply-side professionals touched by this program offer energy efficiency solutions

Customer demand for energy-efficiency services stimulates supply-side professionals to provide solutions/ services

Seed Funding

Advisory Group

RPIP

Energy efficiency professionals' response to consumer request is increased energy efficiency

Energy efficiency increases because supply-side professionals, responding to greater demand, offer consumers more energy efficient solutions

Code Support

State Special Projects

Maintain current code (= still viable) Energy codes are maintained and remain viable

Advisory Group The EBPI is seen as a regional effort Alliance stakeholders see EBPI as a regional effort

Advisory Group A strategy and reason exists to continue the project

EBPI has a promising strategy and there is evidence that there is a reason to continue, as is or modified, based on evaluation findings



The Seed Funding element of EBPI was very limited and the RPIP included little in the way of outreach or services that directly encouraged target audiences to seek out energy efficiency expertise. To find it, the mass media advertising directed them to the website, where they would encounter the Help Desk. The Advisor Service through the Help Desk did provide a means for business decision-makers and supply-side service providers to link with energy efficiency professionals; however, few accessed the service. The Advocacy Program for the design community, which might have stirred more awareness of betterbricks.com, since it was to include direct mail, a speaker series with noted efficiency experts, and a toolkit for helping professionals take action, was postponed because the Design Guidelines were not completed.

2. Awareness of Productivity and How It Relates to Energy Efficiency Increases Among Nonresidential Demand-Side Actors

No change in awareness of the productivity/energy efficiency relationship occurred when overall results of the baseline and follow-up surveys were compared. When asked to name the physical factors that most relate to productivity, the same proportion of new interviewees in both surveys of decision-makers and influential staff mentioned energy efficiency features targeted by betterbricks.com (i.e., good lighting, natural light, and comfortable temperature).

However, for those who had been previously interviewed (and thus primed for the concept that productivity might be related to workspace features), we did find statistically-significant increases in the mention of good lighting systems and comfortable temperature as important to physical workspace productivity. Awareness of the betterbricks.com campaign also influenced awareness, but only in one area: respondents aware of the campaign were significantly more likely to mention comfortable temperature as a key factor in productivity. This suggests direct person-to-person communication may work better than mass media advertising.

3. Availability of Energy Efficiency Professionals Increases

No baseline has been established for the number of energy efficiency professionals in the region. As with the first progress indicator, the lack of effort to involve or affect the supply of or knowledge of supply-side efficiency professionals means that no change should be expected.



4. Energy Efficiency Activities in Buildings Depend Less on Alliance Funding and Move Toward Sustainability

The new construction baseline suggests that a high level of energy efficiency activities is already occurring in new construction, without Alliance funding, due to code requirements, national standards, and ASHRAE standards development. The Alliance-funded advisory services provided a limited number of project-specific supports, as did the Seed Funding effort. The Green Buildings project contributed to the City of Portland adopting higher efficiency standards for its buildings and to establishing a permanently-funded department to oversee sustainable development. These EBPI efforts, though likely resulting in energy savings, are small compared to the non-programmatic region-wide activities. However, these types of efforts may be important as case studies that can help promote even higher efficiency levels than found in the new construction baseline survey.

Based upon interviews with agencies that received code support funding from the Alliance, continued funding for code maintenance, development, and training will likely be needed. Sustained funding for these activities has been difficult for each of the three states that requested and received funding (Idaho, Washington, and Oregon). The agencies report that code compliance and progress would likely drop if these support services did not exist.

5. Customer Demand for Energy-Efficiency Services Stimulates Supply-Side Professionals to Provide Solutions/Services

We found no programmatic effect on increased demand at this time. While three program elements were directly related to this outcome, one of those elements was barely implemented (Seed Funding) and one was not implemented at all (Advisory Group). While the RPIP did increase some awareness of productivity and energy efficiency in buildings, it did not get to the level of increasing demand.

According to interviews with architects through the A+E program and for EBPI, architects in the region report that there has been an upward trend in demand for energy efficiency services for the past five to ten years. The new construction baseline demonstrates that energy efficiency features are being incorporated into the majority of new construction projects in the region, yet all agree that more demand from clients would lead to increased energy efficiency in buildings.



6. Energy Efficiency Increases Because Supply-Side Professionals, Responding to Greater Demand, Offer Consumers More Energy Efficient Solutions

Since we found no evidence of increased demand through program elements for energy efficient solutions, we also found no evidence that the program influenced demand-side professionals to provide more efficiency. (Again, Seed Funding was weak, the Advisory Group non-existent, and the RPIP advocacy program was not implemented, nor was it a primary link to this outcome).

7. Energy Codes are Maintained and Remain Viable

According to interviews with code support agencies in Oregon, Washington, and Idaho, Alliance funding has worked effectively to maintain viable energy codes in those states. Montana did not participate in the EBPI program so we do not have information on their code implementation experience during this Alliance funding period. Without the Alliance funding in Oregon and Washington, energy code support would likely have been severely impaired or non-existent, and in Idaho, services would have been less.

In Idaho, funding fostered better basic services and helped with the preliminary adoption of the International Building Code, which includes stricter residential and uniform non-residential standards. In Oregon, the funding helped sponsor innovative solutions, such as web access to materials and forms; and in Washington it contributed to 50 to 60 specific code improvements. As discussed above, the lack of code support would probably result in lower code compliance (i.e., through less well-trained code officials) and less code development.

8. Alliance Stakeholders See EBPI as a Regional Effort

Meeting this goal was linked to the establishment of an Advisory Group, but this group was not established. The extent to which Alliance stakeholders are aware of the EBPI as a regional effort was not measured, since no explicit programmatic efforts were made in this direction. However, interviews with the Code Support agencies suggest they would like the Alliance to play a stronger regional role and that they didn’t have a sense of the strategy of EBPI overall, especially the betterbricks.com campaign.



9. EBPI Has a Promising Strategy and There Is Evidence That There Is a Reason to Continue, As Is or Modified, Based on Evaluation Findings

Meeting this goal was also linked to the non-existent Advisory Group. However, this evaluation suggests that there are elements of EBPI that are promising in terms of market transformation, including some of the State Special Projects, Code Support, the concept of Seed Funding, and aspects of the RPIP.

CONCLUSIONS

As evidenced above, we were not able to provide much evidence about how EBPI progressed toward several of the short-term outcomes stated in the Logic Model. Clearly, any program logic or theory is likely to have imperfections, and, as stated earlier in this report, measuring EBPI in the short-run is likely to be challenging since it is a complex program, trying innovative things to change the market over a ten to twenty-year timeframe.

Still, the findings about the progress indicators for EBPI point out two very important factors that impeded measurement of progress indicators:

1. The connections of program elements to short-term outcomes are non-existent, weak, overly optimistic, or indirect. Many of the short-term outcomes in the Logic Model depended upon EBPI elements that were not implemented (the Advisory Group) or that were barely implemented (Seed Funding). In addition, some outcomes, while labeled “short-term,” could only be expected to be observed over a longer time horizon (e.g., increased demand due to RPIP efforts). Other outcomes were indirectly connected to elements (e.g., increased offering of energy efficiency services by supply-side professionals due to increased demand.)

2. Measurable goals (qualitative or quantitative) for the level of change that should result from specific program efforts were not set. Thus, it was difficult to accurately assess progress toward indicators. In some cases, baseline data still needs to be gathered (e.g., level of demand for energy efficiency professionals) so that progress could be measured against the baseline. However, gathering such baseline data only makes sense if program efforts are focused on activities designed to affect that baseline.

Although we had difficulties in measuring EBPI progress for the stated outcomes, we still believe there are valuable lessons to be learned from EBPI. These are discussed in Chapter 11.


CHAPTER 11: CONCLUSIONS AND LESSONS FOR EBPI

INTRODUCTION

This chapter will address three questions for the $6.5 million Efficient Building Practices Initiative (EBPI) at the Northwest Energy Efficiency Alliance:

Ø What did EBPI set out to do?

Ø What can we conclude about EBPI’s accomplishments?

Ø What are the major lessons of EBPI?

In answering these evaluative questions for EBPI, we would like the reader to keep the following considerations in mind:

Ø EBPI is a complex program with ambitious goals. Compared to the context in which it is operating and that it is trying to influence – the market for constructing and renovating commercial buildings and the advancement of energy codes – EBPI has had a short life and definitive results should not be expected at this time on many of its aspects.

Ø The short-term outcomes specified for EBPI in the logic model (see Chapter 10) have few specifically measurable activities or results attached to them. This adds to our evaluation challenge since we have few benchmarks against which to compare performance. It also adds to the reader’s challenge as the reader attempts to determine whether the EBPI worked or not.

Ø This evaluation does not, and was never intended to, equally address all components of EBPI. Some aspects of a typical evaluation, such as assessing the program development and decision-making process, were outside the evaluation scope. Furthermore, evaluation activities and priorities changed according to the Alliance's perception of information requirements, especially as the RPIP was launched, and as the EBPI effort evolved during 2000.

Even with these caveats, we believe examining the short life of EBPI yields very useful conclusions and lessons. We have not included recommendations for

Chapter 11: Conclusions and Issues for EBPI


revamping EBPI since the Alliance is developing new strategies to address the commercial building and renovation market that go far beyond modifying EBPI. However, we hope the Alliance will use these conclusions and lessons as it develops the new strategies. In addition, Chapter 12 steps back and takes a larger view of what needs to be considered in transforming the commercial building market, based on EBPI findings and other research.

WHAT DID EBPI SET OUT TO DO?

To address this question, we need to return to the two basic goals underlying the design of the initiative. This is an appropriate time to revisit these goals to see if they remain accurate and adequate for describing what EBPI set out to do. The goals were:

Ø To increase the consumer demand among commercial building users and decision-makers for highly energy-efficient buildings over a twenty-year period.

Ø To enhance the capability of code-related institutions so they can meet increased consumer demand for efficient buildings.

The first goal was largely served by the RPIP betterbricks.com effort. The logic behind this goal was based upon attracting a mass audience to be interested in the need for better buildings that would incorporate improved energy efficient elements. The theory was that if one can attract a mass audience – through advertising and marketing – to the website, and at the website stimulate visitors to be interested in the product (better buildings), then the mass market will eventually demand that product, and products and services will become available to respond to that demand. This type of approach, when applied to achieving a public goal (and hopefully public “good”) such as energy efficiency, and when sponsored by public funds, is often referred to as “social marketing.”

Code Support and State Special Projects activities served the second goal. While these efforts were successful in many ways, few of their activities were aimed directly at meeting increased “consumer demand for efficient buildings,” except in the broad sense of helping to ensure high efficiency buildings can be built under the codes. Meeting consumer demand is not a traditional perspective for code-related agencies. In keeping with their traditional roles, the code-related projects within EBPI were aimed at code maintenance and development, and training and education activities for targeted audiences

Chapter 11: Conclusions and Lessons for EBPI


Neither goal, however, acknowledges the actual and intended supply-side activities (e.g., the Advisor Service, Seed Funding) that were added into EBPI’s design so that there could be a “handshake” between consumers that wanted more energy efficient buildings and energy efficiency professionals who could meet that demand. In addition, Seed Funding and the Advisory Group were meant to help build supply-side capabilities, but these activities had limited or no activities associated with them. These goals also do not take into account the many other actors involved in the commercial development and building process – such as developers, financial backers, real estate professionals, and contractors – who influence consumer demand.

Thus, EBPI goals, as written, only appear to address some of the market elements that are needed to transform the commercial building market, leaving out some that appear critical – particularly more attention to supply-side actors, attention to the development and financial decision-makers who often spearhead new commercial projects, and to the government agencies that influence what types of projects can be built and where.

WHAT CAN WE CONCLUDE ABOUT EBPI’S ACCOMPLISHMENTS?

Chapter 10 used EBPI’s nine short-term outcomes (as listed in the Logic Model) as progress indicators, pointing out the many primary and secondary connections between program elements and intended outcomes. If we boil down the desired short-term outcomes from the logic model, we believe there are five questions that need to be addressed when summarizing EBPI’s accomplishments. These five questions, and our findings vis-à-vis these questions, are provided below:

1. Did EBPI increase awareness among demand-side audiences of the connection between productivity and energy efficiency opportunities in commercial buildings?

About 4% of the demand-side target market became aware of the betterbricks.com campaign. One finding relating productivity to energy efficiency can be attributed to program efforts: those who were aware of the campaign were significantly more likely to mention comfortable temperature as important to productivity.

Participating twice in the surveys (baseline and follow-up) seemed to have more effect on awareness than the advertising. Top-of-mind connections increased for productivity and good lighting systems and comfortable temperature solutions for those who had been previously interviewed. We believe the surveys “primed”



respondents to think about the connection between productivity and workspace features.

2. Did EBPI foster more, and more able, supply-side energy efficiency professionals?

There are two aspects to this question. First, the advocacy program targeted for the supply-side service providers was not launched. Nonetheless, based upon the survey with architects, about 10% of the architects in the region are aware of the program, higher than the 4% for demand-side targets. A few report they plan to use betterbricks.com resources for their own work.

Second, there is no baseline of energy efficiency professionals on which to measure a change, and there were no specific efforts outside of one Seed Funding project to stimulate an increase in energy efficiency professionals. We believe there was no change in this area.

3. Did EBPI help maintain/enhance energy codes in the region?

Yes. Staff continued to maintain the energy codes in Idaho, Oregon, and Washington. Code revisions occurred in Oregon and Washington, and Idaho preliminarily adopted the International Building Code, which includes a more stringent energy code that must be adopted by those who currently have energy codes and by any jurisdictions adopting the IBC.

4. Did EBPI foster self-sufficiency or sustainability in the marketplace (i.e., market transformation)?

Very little effort was exerted in this direction. The Seed Funding component focused primarily on projects needed for RPIP. No projects were formally funded through a Seed Funding RFP, but the Brewery Blocks project may come close to the idea behind seed funding. This project successfully involved a private-sector energy and environmental specialist in a large commercial development. In addition, this specialist is developing a tenant “users” manual, focusing on energy and environmental issues, which may become a model for other commercial projects. These efforts demonstrate that the potential does exist for developing sustainable business opportunities.



5. Will EBPI save energy (implied impact goal)?

Yes. There are energy savings associated with many of the components in EBPI: Code Support, State Special Projects, the projects coming to the Advisor Service at the website, the Brewery Blocks project support, the Design Guidelines, and Future@Work. All of these project activities are likely to result in energy savings that would not have occurred without EBPI.

WHAT ARE THE MAJOR LESSONS OF EBPI?

This section summarizes the important, but program-specific, insights we’ve gained from evaluating particular EBPI elements to date. Larger, crosscutting insights that the Alliance should consider as it plans its strategies for future commercial sector initiatives are discussed in Chapter 12.

The Regional Public Information Program (RPIP)

The RPIP, as embodied in the betterbricks.com campaign was the centerpiece of EBPI. The majority of program and evaluation resources went to developing, implementing, and examining various aspects of that campaign – for instance, the television and other advertising, the website design and use, and the Help Desk. Insights from the RPIP portion of EBPI are discussed below. Most conclusions are based on findings from more than one evaluation activity. (For more detailed findings from individual evaluation activities, please see the Baseline Report, and this report and its appendices.)

1. Embarking on a major mass media and marketing campaign without testing the advertising approaches with the target audiences likely resulted in a less effective campaign.

One of the most important lessons from the RPIP is that mass media approaches and messages need to be tested with target audiences before launch. Substantial work was done to develop the message and to understand what might work with the target markets. Much of this research confirmed previous studies about the need to focus on non-energy benefits, and which non-energy benefits might be a good hook for the markets.

However, the translation of this foundational research into advertising was not tested with the target audiences (using storyboards or mock-ups) prior to production and launch. Clearly, this lack of testing contributed to the campaign being less



effective than it could have been. Results of the usability testing and focus groups revealed audiences often did not understand the messages being conveyed through the ads, weren’t sure about their intent, and, in some cases, responded negatively.

The focus group and usability test results suggest that the television ads need to more explicitly state the brand name and tell people clearly to visit the website. At the same time, audiences want ads to be entertaining, but not obnoxious or to ridicule one solution to promote another.

In addition, testing would likely have revealed the following important insights:

Ø Productivity is a powerful idea, but target audiences did not link it easily to energy efficiency.

Future market research efforts need to focus on the actual content, look, and feel of the materials to ensure that they effectively reach the intended markets and convey the intended message.

The findings in the baseline research, the focus groups, the usability study, the return visitor survey on the betterbricks.com website, and experience of the Advisor Service all suggest that productivity is a hook for the target audiences (even if it is not the only one). However, building users, designers, and developers do not often link productivity to energy efficiency improvements. Improved productivity is more likely to be linked with ergonomics, accessibility, equipment, general comfort, and office layout than to specific energy efficiency components like lighting, temperature control, and airflow.

The follow-up survey with business decision-makers and influential staff, and with architects, however, shed new light on this. The previously interviewed decision-makers and influential staff, and the architects who had participated in A+E, were significantly more likely to notice the betterbricks.com ads and make the connection between productivity and energy efficiency than the others we interviewed. This suggests that the link between productivity workspace improvements can be made more easily if there is more explicit priming of the market so that the audiences are more prepared to receive the messages.

Ø Relying on productivity as the one link to energy efficiency likely limited the strength of the campaign.

The initial research promoted the use of “ME”-oriented benefits of energy efficiency. This research showed that at least two “ME” factors other than productivity –



comfort and health – were likely to “hook” target audiences to being concerned about the physical workspace and, ultimately, about energy efficiency issues. While all three factors presented some challenges, productivity was chosen as the most powerful and was the single focus of the campaign. Yet, as described above, productivity carries with it many meanings and is difficult to make concrete. In addition, productivity is most likely to be a concern of “bosses” not general employees. Further testing of the advertising concepts would have helped clarify the best marketing/advertising propositions to use.

Ø Indirect approaches (i.e., no obvious connection of the ads to energy efficiency or to the Alliance) did not appeal to audiences.

The usability test subjects wanted a clearer statement of betterbricks.com sponsorship and what those sponsors are trying to do in the ads and at the website. Focus group participants strongly preferred direct approaches as well. In particular, focus groups viewed some of the print ads that related energy-consuming elements of the workspace to services being supplied through betterbricks.com as more acceptable than the television ads.

2. General commercial employees are not likely to influence building space decisions and create demand for more efficient buildings. Business decision-makers and members of the design, development, and financial communities are smaller target audiences that likely offer better leverage points.

The theory behind the betterbricks.com mass media campaign was that general employees (along with influential employees and decision-makers) could be rallied to the cause of better, more productive workspaces, which, in turn, would foster certain activities related to energy efficiency. The usability test and the focus group results clearly suggest that general commercial employees do not usually influence building space decisions. Although a significant number of the respondents to the return visitor survey at the website were general employees, they were not represented at all in the project requests or in the projects that were visited by advisors.

On the other hand, supply-side service providers (e.g., architects, contractors) and business decision-makers were the ones who brought projects to the Advisor Service. Website visits and focus groups with business decision-makers revealed that those anticipating workspace changes were even more likely to be interested in betterbricks.com. In addition, the focus group with developers and real estate



professionals showed that this community was likely to include many of the “movers and shakers” in the building industry, and other studies suggest that those who provide capital for buildings are also crucial. These findings all suggest a set of smaller, specialized audiences that need to be targeted as change agents.

3. Given the smaller size and predilections of key audiences (business decision-makers and those in the design and development communities), television advertising is not likely to be the most effective outreach mechanism. Targeted print ads and stories, as well as more personal outreach mechanisms (e.g., peer referral) are likely to have greater effect.

According to results of the follow-up surveys, television advertising did reach a small percentage of decision-makers, influential staff, and architects. In addition, the focus group with developers showed that all had noticed the ads (even if they did not understand them). This suggests that additional and improved advertising would reach more of these audiences, which would, in turn, prompt more to visit the website.

But is this the most effective way to reach and motivate these small audiences, given the shotgun spray of TV advertising? Probably not – television advertising is expensive for reaching such small audiences. In addition, various findings suggest that these target groups are more likely influenced by their peers or by information that appears in sources particular to their industries. To improve EBPI advertising and marketing through the RPIP, the definitions of the size and structure of the market to be influenced would need to be revamped, and outreach mechanisms would need to be carefully selected to reach these markets.

4. Betterbricks.com has drawing power, but involvement was limited. Key audiences voiced interest in betterbricks.com resources and in partnering arrangements.

We found the research respondents in the usability test and focus groups to be interested in betterbricks.com. Decision-makers and influential employees were interested in what betterbricks.com has to offer, especially if they were contemplating new space. The supply-side contacts and developers were particularly interested in the website. Many supply-side service providers who reached the Advisor Service specifically asked how they could partner with betterbricks.com, and those in the focus group saw the potential value in partnering.



Yet, the follow-up survey with business decision-makers and influential staff found only 2% of those who did not participate in the baseline survey were aware of the campaign, with 4% of the entire sample aware of the effort. The involvement with the brand is limited after this first year.

For architects the story was a little different, with about 10% of all architects aware of the campaign. The potential for brand involvement among supply-side service providers is perceived as quite high, with some of those who have accessed the website using it with the clients.

5. To take action, betterbricks.com website users needed more in-depth information of the right kind, and probably other types of help, than the site supplied. Lack of “next step” information and technical assistance hurt program progress.

The Advisor Service had no betterbricks.com branded material to provide users of their service other than the information on the website. The Design Guidelines, key information to the effort for all potential audiences, were not ready. Most usability test subjects wanted more depth than what was on the site. Architects in the follow-up survey noted that the website materials were good to show clients ideas, but did not really help sell or design a project.

A concise, simple level of information is adequate only until the user wants to take action. At that point, the user needs more depth and the right kind of information or assistance. There are many directions that can be pursued to increase the information, such as providing more information on technical topics, budget impacts, supply-side contacts, and insights about the strengths and weaknesses of each of the solutions presented on the website.

To be effective, however, information and assistance needs to be designed with the guidance of target audiences, asking them questions such as: What materials and help (i.e., case studies, design guidelines, design input, contact lists) do you need? What styles and approaches work best? The materials should be tested before final production or site launching to ensure that they do, in fact, meet the needs of different target markets.

Code Support, State Special Projects, Seed Funding

The other major arm of EBPI supported code-related activities, from direct support of agencies developing and maintaining energy codes to projects that advanced new code-related technologies. Conclusions from these efforts are reported below.



Code Support

1. Alliance code support has worked effectively to maintain and advance energy codes in Idaho, Oregon, and Washington. Permanent, sustainable funding for code support activities is a difficult goal to achieve, meaning that ongoing support will likely be needed.

Without the funding in Idaho, Washington, and Oregon, energy code support and maintenance would have been, at the least, impaired and, at the most, non-existent. Over time, the lack of code support would have resulted in lower code compliance (i.e., through less well-trained code officials) and less code development.

If it is accurate that the Alliance primarily funded code support just to be sure that staff with institutional memory were kept at their jobs, the results show the states leveraged the money well beyond its intent. In Idaho, Alliance funding helped with the preliminary adoption of the IBC; in Oregon, it sponsored innovative solutions, such as web access to materials and forms; and in Washington, it contributed to 50-60 specific code improvements (although a few key changes did not occur).

The investment in codes appears to have been a very cost-beneficial way to spend EBPI funds; it appears the codes are not only being maintained, but improved. Code staff in each state appear to be very conscientious in their duties and connected to other relevant entities in their states, and through DOE, to code agencies in the region.

Funding for code support appears to ebb and flow, both at state and national levels. Even though it was a goal of funding to help move code support agencies toward self-sufficiency, agencies did not report much progress toward this end.

2. The Alliance would benefit by having a stronger presence and involvement in codes issues and policy throughout the region, although it would need to determine its appropriate role.

While U.S. DOE has fostered regional coordination among the code agencies with its regional code collaborative, EBPI did not have a regular mechanism to keep in touch with code maintenance and development since it had decided to abandon the Code Advisory Committee element.

A strong Alliance role would be particularly helpful in terms of supporting code changes that the states are trying to achieve, and in helping to resolve code “turf” wars. The state code agencies would like to see the Alliance provide more technical expertise and higher-level policy and political support. In Washington, for instance,



some code language improvements for the non-residential sector did not pass, and, according to several sources, Alliance support was missing and would have been helpful.

A greater Alliance presence would not only keep the Alliance in better touch with code progress or threats, it would also be a credit to the organization. The Alliance is well respected by code agencies in the region and could increase its influence by being more connected. However, the Alliance will need to make a strong commitment, given the complexity of the codes and the differences among the states.

State Special Projects and Seed Funding

1. State Special Projects, and the Brewery Blocks project under Seed Funding, have achieved specific, valuable – but often invisible – accomplishments.

The State Special Projects have met the Alliance goals of providing innovative code support, fostering voluntary actions beyond code, and building partnerships in the building industry. The Brewery Blocks project included a documented environmental approach in the building design and implementation, and it created a product (the tenant manual) that can be emulated by other developers.

Target audiences within the commercial building market consistently report they would like more “hard evidence” about the efficacy of incorporating energy efficiency features into buildings. These projects offer many opportunities for greater exposure throughout the region to interested parties such as code agencies, governmental agencies concerned with building and construction, and the private sector. The Green Buildings and the Brewery Blocks projects are good examples of how these efforts can become case studies and examples for others to follow. The Alliance can get more “bang for the buck” from these projects by promoting their results and benefits.

2. Flexible grant programs like State Special Projects and Seed Funding can assist many different activities proposed by an array of entitles that may not fit into other larger, more focused programs (e.g., Code Support, RPIP).

These smaller efforts in research, building technology, and code development are crucial to “push the envelope” and offer specific, measurable case studies and solutions. Taken together, they can be used to promote the broader objective of market transformation.



The New Construction Baselines

1. The new construction baselines provide valuable information and point to opportunities in new construction, but their benefits cannot be realized until the findings have been more widely disseminated and applied.

At their core, the baselines provide an important benchmark against which future efforts to change building construction practices can be assessed. They can also be used to guide programmatic initiatives, directing planners to the areas of greatest need, and reminding us that much has already been achieved. Finally, a good deal of market information, based upon interviews with market actors, is embedded in these studies, and these findings should be factored into program design.


CHAPTER 12: SYNTHESIS

INTRODUCTION

So far in this report we have largely constrained ourselves to examining specific elements of EBPI. Specifically, we focused on how well they worked or did not work within the context of how the initiative was defined over the contract period. Such an approach, while necessary to understand and improve program elements, does not adequately answer the larger question of:

Ø What market theories and intervention strategies provide the best way to achieve market transformation in the commercial building market?

In this chapter we will explore this question by discussing:

Ø How can commercial building market initiatives create change?

Ø What target markets should be pursued with what strategies?

Ø What market change indicators should we look for?

We will draw upon the EBPI evaluation results and upon other recent research that addresses market transformation and understanding commercial building markets. The answers we provide to these questions will not be definitive, but we hope to suggest factors that need to be considered as the Alliance revamps its initiatives to create greater demand for energy efficient buildings in the commercial marketplace.

HOW CAN COMMERCIAL BUILDING INITIATIVES CREATE MARKET CHANGES?

Theory Underlying Market Change

We focus first upon theory because we agree with others that strong market and program theories stand behind effective programs and program evaluations (Weiss, 1997; Kunkle and Lutzenhiser, 2001). When talking about theory we do not mean lofty ideas; quite the contrary, we mean down-to-earth descriptions of what happens, and why, in the real world we are trying to influence. A market theory tries to describe how market forces and actors interact to deal with a product or service – in this case, commercial building construction and renovations. A program

Chapter 12: Synthesis


theory describes a set of interventions designed to influence the market in a particular direction – in this case, toward greater energy efficiency.

We are particularly interested in exploring market and program theories because we believe that weaknesses in the theoretical underpinnings of EBPI affected both its effectiveness and our evaluation efforts. In addition, new program strategies that the Alliance develops for intervening in the commercial building marketplace would benefit from a clearer delineation of market and program theories. These theories can then provide a litmus test to see if program elements make sense and can suggest what program indicators should be measured.

Energy efficiency programs typically have been organized around one of four approaches: market transformation, resource acquisition, research and development, and infrastructure. The first two directly intervene in the marketplace, while the latter two provide support.

Market transformation programs try to produce permanent, sustainable change in practices among market actors by intervening in the marketplace. Alternatively, many efficiency programs have been aimed at resource acquisition, or encouraging short-term interventions to get conservation resources quickly; these programs do not expect permanent market changes. Research and development programs provide data and products to support both market transformation and resource acquisition programs. Finally, infrastructure programs provide administrative and structural support to the other three types of efforts.

EBPI Theories In A Nutshell

We assume EBPI’s overall goal is market transformation in the commercial building and renovation marketplace: “ . . .that is, a policy objective of encouraging or inducing social, technological and economic change in the direction of greater energy efficiency” (Blumstein, Goldstone and Lutzenhiser, 2000) geared to produce lasting benefits in the marketplace and which do not require continuing support.

EBPI’s approach to transforming the market was to develop a multi-faceted approach, largely divided between a goal of creating greater “demand” for more efficient buildings (Regional Public Information Program, or RPIP), and a goal to ensure energy codes were maintained and improved (Code Support, State Special Projects, etc.).

RPIP’s market theory was that building users are a mass market that could be the drivers of demand for better buildings. It assumed that supply services (such as



architects, engineers) would follow the demand created by business employees, influential staff, and decision-makers. It also posited: “Beware of the Supply-Side.”

RPIP program theory proposed that the right message “hook” in a mass media campaign, along with supporting information (e.g., Design Guidelines) and services (e.g., the Advisor Service), could activate demand. The campaign focused on “productivity” as the hook, and did not emphasize energy efficiency. RPIP’s strategy was to create a brand – betterbricks.com – and provide branded information and services, particularly through a website. Belatedly, the RPIP also intended to facilitate the “handshake” between demand and supply through advocacy and information to supply-side actors.

Code-related activities did not intervene actively in the commercial marketplace, but they were meant to support market transformation. The program goals were to make sure basic energy codes were maintained, and, beyond that, to improve codes and to spur new activities and technologies that would take efficiency beyond current code. The program theory said that by supporting code agencies, funding special projects to enhance code language, adoption, or technologies, and supporting a regional Code Advisory Group, new and renovated commercial buildings would meet, or exceed, energy code requirements, and, in turn, meet increased demand due to RPIP.

As is evident in these descriptions, the two major EBPI program theories were not well integrated. This made it difficult to articulate an overall program logic and to manage and connect up program pieces.

EBPI Theories Revisited

The Alliance wanted to produce lasting change, through EBPI, in the commercial building marketplace. Kunkle and Lutzenhiser (2001) raise three mechanisms required for change in this market: creating value for energy efficiency; institutionalizing demand; and making the supply of energy efficiency routine. We will use these change criteria to assess the viability of EBPI’s theories.

Did EBPI help make energy efficiency more relevant and valuable in the marketplace?

We do not believe that EBPI’s public information program made energy efficiency more relevant in the marketplace, although that was certainly its intent. The RPIP did not conceive of the marketplace as a dynamic set of market actors operating within a powerful context of “economic, social/cultural, and political” conditions and



it did not acknowledge the cultural norm that: “Buildings are Investments” (as described by Kunkle and Lutzenhiser, 2001).

Rather, RPIP focused on building users, where only a small proportion – influential staff and decision-makers – are generally empowered to make space decisions. The RPIP strategy largely left out other powerful actors in the building industry, such as developers, real estate professionals, designers, builders, communities, and regulators. It also believed it could provide a “silver bullet” of motivation through mass advertising, and a limited amount of in-depth information and hands-on assistance.

The RPIP hoped to create greater value for energy efficiency among a mass market of building users by linking it to increased worker productivity, which, in turn would raise demand for more productive buildings including improved energy efficiency. However, the productivity-efficiency connection was not explicitly made in the advertising campaign, and the small proportion of audience members who noticed the ads didn’t easily make the link. Only the few who reached the website saw this connection explicitly made, and even fewer asked for further energy efficiency assistance.

Raising the awareness of the relevance of energy efficiency among all building users, in the mass market sense, is a difficult and expensive ambition, and not likely to have much impact, since research shows that the majority of building users (general employees) have little ability to influence space changes. In addition, relying on only one advertising hook – productivity – limited market relevance for users. Other potentially relevant connections just for the building user – such as being “green,” increasing comfort – and for owner/users – controlling energy costs, better building management, increased investment value, or marketability of the building – were not used.

On the code support and advancement side of EBPI, the relevance of energy efficiency did increase for specific communities and actors. For instance:

Ø The Green Buildings project in Portland exposed many market actors to the relevance of efficiency through its wide community involvement and creation of hard evidence for cost-effectiveness. The new city policy for green buildings will extend this influence.

Ø In the Brewery Blocks project, integration of an environmental expert with the design team made energy efficiency more relevant by linking it to tangible improvements in the marketability and management of the building.



Ø In Idaho, rural communities adopted energy codes for the first time, and a more stringent energy code is expected to be adopted statewide. The appeal of these codes came from citizens benefiting and being more empowered by a stronger building code. (Notably, “environmental protection” and the international backing of the IBC were messages to be avoided in Idaho.)

Did EBPI help increase institutionalized demand for energy efficiency among specific market actors?

Without increased relevance for energy efficiency, demand is unlikely to increase in the market. RPIP’s marketing treated building users as a mass audience. It made few efforts to reach specific actors within this audience who might be more receptive to demanding more efficient buildings – for instance, government users (e.g., cities like Portland or Seattle), owner-users, and build-to-suit users. The shotgun RPIP strategy was not likely to result in what Kunkle and Lutzenhiser call “concrete expressions of willingness to act in particular ways by concrete actors on the ground.”

The RPIP did not determine which specific market actors, within more general target audiences, might be most likely to demand more efficiency. Kunkle and Lutzenhiser suggest “progressive regulators, large institutional users, vertically integrated property developers and managers, institutional investors, and participants in build-to-suit projects. Each of these actors has interests that create opportunities for institutionalizing the demand for energy efficiency through regulatory incentives, work environment standards, investment requirements, unique user interests, and long-term property value.”

Where energy efficiency was made relevant to market actors (e.g., through the Green Buildings Project that engendered the Office of Sustainability in Portland or through the Brewery Blocks projects where developers, designers, and users better embraced the worth of efficiency approaches), demand is likely to become more institutionalized.

Did EBPI help incorporate the supply of energy efficiency into the standard routines of the building industry?

The RPIP did not make inroads within the design/development process. It is trying, through the development of the Design Guidelines, to illustrate how energy efficiency approaches might be made more routine, but these efforts are not complete. The Advisor Service also may offer a mechanism to take advantage of



existing interests in the marketplace, but to date this service has not gotten much use.

On the codes side of EBPI, specific projects may help make energy efficiency choices more routine. The demand controlled ventilation technology project hopes to make DCV a standard industry practice without a need to become code, because the industry will see its value and convenience. According to the project manager, the oversight and endorsement of a national technical advisory committee is crucial in making DCV routinely chosen. The Green Buildings project will make efficiency choices more routine for Portland city buildings.

Recommendations

We suggest generic versions of the three questions above be applied to any new theories and strategies of market transformation that the Alliance considers for the commercial buildings sector. These are broad market transformation criteria, validated by research, evaluation, and experience. These criteria need to be used to assess the adequacy of proposed market and program theories.

The market theory for change needs to describe how it will foster energy efficiency being made more relevant, more institutionalized in terms of demand, and more routine in terms of supply.

The program theory for change needs to demonstrate how and why its strategies will work in support of changing the market toward greater energy efficiency (and any other goals). This analysis needs to describe: the specific market actors to be targeted; the program mechanisms; the reasons why actors would be interested in change, in general, and using the proposed mechanism, in particular; the best ways to reach market actors; the intersection of market actors with business types; and indicators of how change will measured in the market.

WHAT TARGET MARKETS SHOULD BE PURSUED WITH WHAT STRATEGIES?

Introduction

In planning strategies for the commercial buildings and renovation market, we believe:

Ø There are several useful approaches for segmenting the market, depending upon how you want to view it. Segmentation approaches,



however, need to be well defined (e.g., market actors, business types, areas of market opportunity, methods of influence).

Ø It is essential to see the intersections among market segmentation approaches, so that the complexity of the market is reinforced and so that program elements can be integrated.

Ø A system that considers the intersection of market segments, as well as other factors important to the funding agency, would be useful for choosing the best program approaches.

Segmenting the Market

One way to organize the commercial market is around business or building types – for instance, small retail, groceries, schools, offices, etc. This perspective is important to gauge the amount of activity in the market and the opportunities for influencing greater efficiency. In addition, our research shows that market actors identify themselves as part of particular industries, often do work only in particular business segments, and are more influenced by communications and services that are endorsed through their industries.

Kunkle and Lutzenhiser’s recent draft report for the Alliance (2001, p. vii) presents a table taken from the perspective of “areas of opportunity.” This table first lists trends they have found relevant to the building industry (e.g., the movement toward more green and sustainable buildings) and then shows how the opportunities intersect with key market actors, existing activities, and mechanisms to reach or affect the market (e.g., marketing, guidelines).

From a market-change perspective, segmentation by market actors is a key activity, since it is people who make decisions and take actions, it is people who organize themselves into groups, and it is people who need to be reached, appealed to, and served. One of the strengths of EBPI is that it revealed a good deal about the nature of certain potential market actors (even if it didn’t do well at choosing the best market segments). Other recent research (Kunkle and Lutzenhiser, 2001; Heschong-Mahone, 2000; Reed, Oh, and Hall, 2000) has illuminated the types of and roles of market actors.

Recommendations

We believe further segmentation and integration will help in program planning. We would like to see more work on market actors, including:



Ø Segmenting market actors into smaller, more precise groups (which can then be aligned through common interests and mechanisms). For instance, the EBPI research suggests that the terms demand-side and supply-side are not good umbrellas to define market actors in a dynamic environment where roles can change. Architects are suppliers when a client purchases their services, but almost never supply buildings to the general market place. Developers supply buildings to the market, but they demand design/build services from architects and builders.

Ø Using Kunkle and Lutzenhiser’s (2001) list of market actors as a starting point (major and sub-categories) for finer segmentation, but using a critical eye and market experts to review and revise this list. Bear in mind that to reach market actors efficiently, it’s best if they belong to a reachable group with which they identify and share common interests. Thus, a key question to ask is: Do actors see themselves as aligned with the organizational forms, as listed, or with some other attribute (e.g., professional kinship or business industry)? The actor categories Kunkle and Lutzenhiser suggest are:

• Providers of Capital – Investors (Financial Institutions, Institutional/Pension Funds, Financial Markets, Wall Street), Owner-Occupied, Public Owners, Private Owners

• Developers – Build-to-Sell, Build-to-Hold, Build-to-Suit

• Design and Delivery – Design/Bid/Build, Design/Build, Design-Assist/Construction Manager, Hybrids

• Community/Political/Regulatory Interests – Pro-Development, Progressive, Restrictive

• Real Estate Services Providers – Marketing/Sales, Leasing, Investing, Management/Operations

• Users of Buildings – Lease, Owner Occupied

Ø Development of consistent terms and definitions and/or descriptions of these market groups and their roles. If overlap and inconsistencies exist, acknowledge it. The lack of agreement about these terms creates considerable confusion.

In addition, we believe the market actor categories need to be integrated with other types of segmentation and information. To accomplish this, we recommend:



Ø Doing a “first cut” determination of market actor segments to pursue. This exercise would list market actors, describe key market intelligence (or lack thereof), and show key intersections with other segmentation approaches. Table 46 illustrates this type of approach with “Providers of Capital.” Although our analysis grid would benefit from further refinement, and better information, we chose this category because investors may be the least understood or pursued market actor in the industry as far as energy efficiency improvements are concerned. On the table, market intelligence is addressed by these topics:

• Key market viewpoints

• Key business types

• Opportunities for influence (complementary industry trends)

• Likelihood of successful intervention

• Level of influence in commercial building market

• How to reach

• Current activities

• Other ideas

• Champions or projects that might be approached

• Need for primary research

• Pursue further? And, if so, how to pursue (based upon positive interest, ability to influence and reach the market actor, no major barriers, lynchpin status)?

Ø Doing a fuller analysis of chosen target actors, expanding on market intelligence and program strategies, and then checking with market experts.



Table 46: Market Actor Analysis (for illustration only)



MARKET INTELLIGENCE

FINANCIAL



MARKETS/ WALL

STREET

REAL ESTATE

INVESTMENT

TRUSTS


Key Market Viewpoints


Low interest in efficiency


Government sets example Efficiency/ greenness per

se

Buildings are investments Desire for productive

workforce

Good corporate citizens

Business Types ? ? ? ? Offices, schools, hospitals (local, state, federal

government?)

Offices (local, regional, national, international

firms)

Opportunities for Influence


Better income stream, return

Green Buildings


Green buildings


Work environment

Likelihood of Successful Intervention

Low? Low? Low? Low? High Medium

Level of Influence High, initial decisions made here, big picture High influence investors High to medium, depending on visibility of

owner

Continued





MARKET INTELLIGENCE

FINANCIAL



MARKETS/ WALL

STREET

REAL ESTATE

INVESTMENT

TRUSTS


How to Reach Professional groups, industry pubs, person to person, industry leaders, standards/regulation

Professional groups, industry pubs, peer

influence, press stories, community interests,

public policy, regulation

Professional groups, industry pubs, press stories,

public recognition, regulation

Current Activities ? ? ENERGY STAR Buildings?

ENERGY STAR Buildings?

Green Buildings/ LEED/ Earth Advantage

Brewery Blocks

Other Ideas ? Socially Responsible Investing?



? ?

Champions, Leads, Projects

? ? ENERGY STAR

Buildings investment

tool?

ENERGY STAR Building

Partners?

LEED Buildings, Requirements to build to

LEED standards?

Owners that are using LEED?

Need for Primary Research

Maybe Yes Yes Yes Probably not Yes, focus on large institutional users

Pursue Further Maybe research

Yes, research Yes, research Yes, research Yes, high priority Yes, high priority

* Question marks signal lack of knowledge or barriers



WHAT MARKET CHANGE INDICATORS SHOULD WE LOOK FOR?

Market change indicators can only be determined once the market and program theories are fleshed out. However, indicators would need to be consistent with the overall market change criteria we suggested applying as litmus tests to theories: increased relevance of, increased demand for, and increased supply of energy efficiency in the commercial buildings marketplace. (See the excellent discussion in Kunkle and Lutzenhiser 2001.)

Depending on the program strategy, indicators need to look for changes in the market that show how the energy efficiency message has value in the market place or how supply and demand for energy efficiency is becoming more institutionalized in market practices. In addition, these researchers also suggest looking at energy efficiency indicators to see if more energy efficiency has been built into new buildings or retrofitted into existing ones in the market place. Follow-ups to the commercial Baseline Survey undertaken through EBPI are one approach to track progress in energy efficiency. Expanding on existing data collection efforts such as the Commercial Buildings Energy Consumption Survey (Energy Information Administration) would provide important information on broader trends in building energy efficiency. These evaluation and research activities improve our understanding of the market and support future program development activities.

REFERENCES FOR CHAPTER 12

Blumstein, Carl, Seymour Goldstone and Loren Lutzenhiser. 2000. “A Theory-Based Approach to Market Transformation,” Energy Policy 28:137-144.

Heschong-Mahone Group 2000. Nonresidential New Construction Market Assessment and Evaluation: Market Transformation Barriers and Strategies Study. Los Angeles: Southern California Edison.

Johnson, Jeff and Steven Nadel 2000. “Commercial New Construction Programs: Results from the 90’s, Directions for the Next Decade.” Proceedings of the ACEEE 2000 Summer Study on Energy Efficiency in Buildings, 4.187-203. Washington, DC: American Council for an Energy Efficient Economy.

Kunkle, Rick and Loren Lutzenhiser, 2001. New Commercial Office Building Markets: Developing Strategic Interventions. Draft report. Portland: Northwest Energy Efficiency Alliance.



Kunkle, Rick and Loren Lutzenhiser, 2001. “An Improved Model for New Commercial Office Building Markets: Implications for Market Research and Evaluation.” Paper to be published and presented at the 2001 International Energy Program Evaluation Conference. Salt Lake City, Utah.

Lutzenhiser, Loren, Nicole Woolsey Biggart, Rick Kunkle, Thomas Beamish and Thomas Burr 2001. New Commercial Buildings: Market Transformation Potentials. University of California, Berkley: California Institute for Energy Efficiency.

Parker, Gretchen, Mark Chao and Ken Gillespie 1999. “Energy-Related Practices and Investment Criteria of Corporate Decision Makers.” [ ]

Reed, John, Andrew Oh, and Nicholas Hall 2000. “The Structure and Operation of the Commercial Building Market.” Proceedings of the ACEEE 2000 Summer Study on Energy Efficiency in Buildings, 4.267-283. Washington, DC: American Council for an Energy Efficient Economy.

Weiss, C. H. 1997. Evaluation: Methods for Studying Programs and Policies. New York: Prentice Hall.



Final Report

Market Progress Evaluation Report #1

EFFICIENT BUILDING PRACTICES INITIATIVE

Volume 2: Appendices

Funded By:

Submitted To:

Jane Gordon, Ph.D.


Prepared By:

DETHMAN & TANGORA h




July 2001

EBPI – MPER #1 DETHMAN & TANGORA h A - i

TABLE OF CONTENTS

APPENDICES

APPENDIX A: BETTERBRICKS.COM QUESTIONNAIRE............................................................. A-1

APPENDIX B: ARCHITECTURE + ENERGY (A+E) 2001 ARCHITECT INTERVIEW GUIDE...........B-1

APPENDIX C: DRAFT FOCUS GROUP REPORT FOR RPIP’S BETTERBRICKS.COM CAMPAIGN

EXECUTIVE SUMMARY................................................................................................... C-I Introduction and Goals ........................................................................................... C-I Methods ................................................................................................................C-II

Key Findings and recommendations .....................................................................C-III

CHAPTER 1 – INTRODUCTION .......................................................................................C-1 The Efficient Buildings Practices Initiative and the Regional Public Information Program .................................................................................................................C-1 The betterbricks.com Information Campaign...........................................................C-1 Focus Group Goals .................................................................................................C-2 Focus Group Methods .............................................................................................C-3

CHAPTER 2 – PERCEPTIONS ABOUT WORKSPACE DESIGN ISSUES, TRENDS, AND ROLES ................................................................................................................C-5

Mover and Non -Mover Groups ................................................................................C-5

CHAPTER 3 – REACTIONS TO TELEVISION ADVERTISING.......................................C-11

CHAPTER 4 – REACTIONS TO PRINT ADVERTISING..................................................C-15

CHAPTER 5 – THE BETTERBRICKS.COM BRAND .......................................................C-17 Adjective Choice ...................................................................................................C-17 What will you find at website? ..............................................................................C-18 Description of betterbricks.com .............................................................................C-18

CHAPTER 5 – INFORMATION SOURCES AND FINAL ADVICE ...................................C-21 Information Sources .............................................................................................C-21 Final Advice.........................................................................................................C-21

APPENDIX: FOCUS GROUP MATERIALS ....................................................................C-23 Betterbricks.com Business Influentials / Decision-Makers Focus Group Discussion Guide ...................................................................................................................C-25 Betterbricks.com Real Estate Developers / Professionals Focus Group Discussion Guide ...................................................................................................................C-33

Table of Contents

DETHMAN & TANGORA h EBPI – MPER #1 A - ii

APPENDIX D: DRAFT BETTERBRICKS.COM WEBSITE USABILITY REPORT

TABLE OF CONTENTS.................................................................................................... D-3

EXECUTIVE SUMMARY...................................................................................................D-I Introduction and Goals ...........................................................................................D-I Methods .................................................................................................................D-I Key Findings.........................................................................................................D-II

CHAPTER ONE – INTRODUCTION, PURPOSE, AND METHODS.................................. D-1

Methods ................................................................................................................ D-1 Organization of the Report .................................................................................... D-3

CHAPTER TWO – USABIL ITY ISSUES AND RECOMMENDATIONS ............................ D-5 Communication of Web Site Purpose ..................................................................... D-5 Organizational Issues ............................................................................................ D-5 Navigational Issues............................................................................................... D-7 Content Issues....................................................................................................... D-8 Stylistic Issues ...................................................................................................... D-8 Advisor Area Issues ............................................................................................... D-9 Field Study Issues ................................................................................................D-10

CHAPTER THREE – RESULTS OF TASKS AND ANSWERS TO QUESTIONS ..............D-13

APPENDICES Appendix A: Table of Usability Problems ..............................................................D-21 Appendix B: Participant Comments .....................................................................D-31 Appendix C: Usability Test Procedures ................................................................D-37 Appendix D: Background on Human Computer Interfaces ....................................D-43

APPENDIX E: FINAL EBPI CODES SUPPORT PROCESS EVALUATION INTERVIEW GUIDE ......E-1

APPENDIX F: STATE SPECIAL PROJECTS PROCESS EVALUATION INTERVIEW GUIDE............ F-1

APPENDIX G: FINAL BASELINE EVALUATION REPORT EBPI – EXECUTIVE SUMMARY..........G-1

BACKGROUND ................................................................................................................ G-1

KEY FINDINGS................................................................................................................ G-2 Baseline Assessment Of Commercial Workers ....................................................... G-2 Baseline Assessment of Commercial Architects and Developers............................. G-4

CONCLUSIONS AND ISSU ES ......................................................................................... G-6 1. The BetterBricks.com message is competing with other messages..................... G-6 2. BetterBricks.com messages may not currently attract developers. .................... G-6 3. Primary target audiences prefer news and prime time TV to sports. ................. G-7 4. Preferences for information sources may vary by target audience. .................... G-7

EBPI – MPER #1 DETHMAN & TANGORA h APPENDICES

APPENDICES

Appendices

DETHMAN & TANGORA h EBPI – MPER #1 APPENDICES

EBPI – MPER #1 DETHMAN & TANGORA h APPENDIX A

Appendix A: betterbricks.com Questionnaire (Influential and Decision-Maker Follow-up Survey)

Appendix A: betterbricks.com Questionnaire

DETHMAN & TANGORA h EBPI – MPER #1 APPENDIX A

EBPI – MPER #1 DETHMAN & TANGORA h A - 1

BETTERBRICKS.COM QUESTIONNAIRE

TZONE Respondent's time zone N= .............................................................................................................................................. 1023 100% Mountain ................................................................................................................................5 134 13% Pacific .....................................................................................................................................6 886 87%

SIC SIC

STATE state N= .............................................................................................................................................. 1023 100% Idaho..................................................................................................................................... ID 95 9% Montana.............................................................................................................................MT 170 17% Oregon ................................................................................................................................OR 326 32% Washington ......................................................................................................................WA 432 42%

TYPE N= .............................................................................................................................................. 1023 100% Previous Decision Maker....................................................................................................1 202 20% Previous Influencer...............................................................................................................2 251 25% New .........................................................................................................................................3 570 56%

COMP Company name

CONT Name of person previously interviewed.


DETHMAN & TANGORA h EBPI – MPER #1 A - 2

SIZE size of business N= .............................................................................................................................................. 1023 100% 20-49 employees ...................................................................................................................4 598 58% 50-99 employees ...................................................................................................................5 252 25% 100-249 employees...............................................................................................................6 129 13% 250-499 employees...............................................................................................................7 19 2% 500-999 employees...............................................................................................................8 15 1% 1000+ employees ..................................................................................................................9 10 1%

INTRODUCTION FOR COMPANIES WE HAVE NOT PREVIOUSLY INTERVIEWED FROM COMPANY NAME: <Comp > Hello, I'm _____ of Gilmore Research Group. We are conducting a brief study for a nonprofit organization focused on improving the quality of the workspace. I need to reach a person in your organization who makes or influences decisions about the space where your employees work. Who would that be at this location? WHEN CORRECT PERSON ON LINE: Hello, I'm ______ of Gilmore Research Group. We are conducting a brief study on what businesses want for the space where their employees work. Are you the person in your organization who makes or influences decisions about employee work space at this location? IF NO, ASK TO SPEAK TO THAT PERSON IF NEEDED: This is not a sales call, it's a research project for a nonprofit organization focused on improving the quality of the workspace. 51 Yes, continue 65 Decision not made (WA OR ID MT) - THANK AND TERMINATE 1 RECORD PERSON'S NAME 1 RECORD PERSON'S TITLE

INT02 => INT05 if TYPE=1-2

N= .............................................................................................................................................. 570 100% Continue ...............................................................................................................................51 564 99%

QX1 N= .............................................................................................................................................. 570 100% Record Name .........................................................................................................................1 O 570 100%

QX2 N= .............................................................................................................................................. 570 100% Record title .............................................................................................................................1 O 570 100%



INTRODUCTION FOR PEOPLE WE INTERVIEWED IN BASELINE WHO SAID WE COULD CALL BACK COMPANY NAME: <Comp > PERSON TO ASK FOR: <CONT > I'm_____of Gilmore Research Group. We interviewed_______last Spring for a non profit organization focused on improving the quality of the workspace. He/she gave us permission to call back should we do further research. IF PERSON PREVIOUSLY INTERVIEWED NO LONGER WORKS THERE/REFUSES, PRESS F1 AND CODE AS 52 TO GO TO NEW INTRO.. WHEN CORRECT PERSON ON LINE: Hello, I'm ______ of Gilmore Research Group. Last Spring we interviewed you for a study sponsored by a non profit organization on what businesses want for the space where their employees work. You gave us permission to call you should we conduct additional research. I'd like to take a few minutes of your time to ask you some questions. IF NEEDED: This is not a sales call, it's a research project for a nonprofit organization focused on improving the quality of the workspace. 51 Yes, continue 52 Continue with new person

INT05 => QA if TYPE=3

N= .............................................................................................................................................. 453 100% Continue ...............................................................................................................................51 => QA 420 93% Continue with new respondent .........................................................................................52 29 6%



INTRO FOR NEW PERSON WHEN PREVIOUSLY INTERVIEWED PERSON NO LONGER WITH COMPANY COMPANY NAME: <Comp > Hello, I'm _____ of Gilmore Research Group. We are conducting a brief study for a nonprofit organization focused on improving the quality of the workspace. I need to reach a person in your organization who makes or influences decisions about the space where your employees work. Who would that be at this location? ASK TO SPEAK TO THAT PERSON. WHEN CORRECT PERSON ON LINE: Hello, I'm ______ of Gilmore Research Group. We are conducting a brief study on what businesses want for the space where their employees work. Are you the person in your organization who makes or influences decisions about employee work space at this location? IF NO, ASK TO SPEAK TO THAT PERSON IF NEEDED: This is not a sales call, it's a research project for a nonprofit organization focused on improving the quality of the workspace. 51 Yes, continue 65 Decision not made (WA OR ID MT) - THANK AND TERMINATE 1 RECORD PERSON'S NAME 1 RECORD PERSON'S TITLE

INT06 N= .............................................................................................................................................. 32 100% Continue ...............................................................................................................................51 32 100%

QX3 N= .............................................................................................................................................. 32 100% Record Name .........................................................................................................................1 O 32 100%

QX4 N= .............................................................................................................................................. 32 100% Record title .............................................................................................................................1 O 32 100%

QA What is the approximate number of people who work at this location?

N= .............................................................................................................................................. 1023 100% 0-19..........................................................................................................................................0 0 0% 20-49 .......................................................................................................................................1 546 53% 50-99 .......................................................................................................................................2 278 27% 100-249 ...................................................................................................................................3 119 12% 250-500 ...................................................................................................................................4 39 4% Over 500.................................................................................................................................5 38 4% Don't know/Not sure.............................................................................................................6 3 0% Refused...................................................................................................................................7 0 0%



QB Does your company own or lease your employee work space? N= .............................................................................................................................................. 1023 100% Own .........................................................................................................................................1 652 64% Lease/Rent..............................................................................................................................2 354 35% Don't know/Not sure.............................................................................................................3 16 2% Refused...................................................................................................................................4 1 0%

QC Which of the following statements best describes your position within your company? Would you say. . . N= .............................................................................................................................................. 1023 100% You're the one in charge of making decisions about employee work space company-wide..................................................................................................................1 187 18% You play a key role in deciding about the kinds of space your company uses ..........2 382 37% You have a significant amount of unofficial influence on these issues.......................3 226 22% You are often asked for your opinion about these issues but don't make the decisions............................................................................................................................4 228 22% You have no influence and are not asked for your opinions on these issues..............5 0 0% ------------------------------------------------............................................................................. 0 0% Don't know/Refused.............................................................................................................6 0 0%

QUCHK check quota N= .............................................................................................................................................. 1023 100% Both quotas open...................................................................................................................1 1008 99% decision maker closed..........................................................................................................2 15 1% influence closed.....................................................................................................................3 0 0%

SETA SET WHO $E N= .............................................................................................................................................. 1023 100% Decision maker......................................................................................................................1 523 51% Influence.................................................................................................................................2 500 49%

26: Q1A How old would you guess your building is. . . N= .............................................................................................................................................. 1023 100% Less than 5 years old ............................................................................................................1 84 8% Between 5 and 20 years .......................................................................................................2 303 30% Older than 20 years...............................................................................................................3 611 60% Don't know/Not sure.............................................................................................................4 24 2% Refused...................................................................................................................................5 1 0%



Q2A And, about how much square footage would you say your company occupies? IF DON'T KNOW/NOT SURE, SAY; Just give me your best estimate.

N= .............................................................................................................................................. 1023 100% Under 10,000 square feet...................................................................................................01 221 22% 10,000 to just under 25,000 square feet ..........................................................................02 237 23% 25,000 to just under 50,000 square feet ..........................................................................03 151 15% 50,000 to just under 100,000 square feet ........................................................................04 78 8% Or 100,000 square feet or more ........................................................................................05 121 12% .................................................................................................................................................... Other (SPECIFY): ..............................................................................................................97 O 0 0% Don't know/Not sure...........................................................................................................98 215 21% Refused.................................................................................................................................99 0 0%

Q3A The remaining questions will just take a few minutes. Are you planning to make, or are you in the process of making, any of the following changes in your work space at this location. . .

N= .............................................................................................................................................. 1023 100%

Q3AA (Are you planning to make, or are you in the process of making, any of the following changes in your work space at this location. . . )

Renovating your current space?

N= .............................................................................................................................................. 1023 100% Yes...........................................................................................................................................1 304 30% No ............................................................................................................................................2 705 69% Don't know/Not sure.............................................................................................................3 14 1% Refused...................................................................................................................................4 0 0%

Q3AB Constructing your own building?




Q3AC Moving to a new leased space?


Q6A How large an effect do you think the physical workspace has on employee productivity? Would you say it has a . . . N= .............................................................................................................................................. 1023 100% Large effect ............................................................................................................................1 626 61% Some effect ............................................................................................................................2 363 35% Or no effect ............................................................................................................................3 32 3% -------------- ............................................................................................................................... 0 0% Don't know/Not sure.............................................................................................................4 2 0% Refused...................................................................................................................................5 0 0%



When you think about your organization's physical space, what factors do you feel contribute the most to employee productivity? IF RESPONDENT MENTIONS COMFORT, PROBE: What do you mean by that? PROBE UP TO 5 MENTIONS 01 RECORD COMMENTS 98 Don't know/Not sure 99 Refused DID THE RESPONDENT MENTION: 1 Good lighting 2 Natural light/windows/daylight 3 Heat/Cooling comfort/Temperature 4 None of the above

Q7A1 N= .............................................................................................................................................. 1023 100% RECORD COMMENTS ...................................................................................................01 O 0 0% Good lighting.......................................................................................................................02 N 302 30% Natural light/Windows/Daylight......................................................................................03 N 119 12% Heat/Cooling comfort/temperature ..................................................................................04 N 175 17% Low noise levels ..................................................................................................................05 N 72 7% Technology or other machinery working well with no breakdowns/good equipment ......................................................................................................................06 N 32 3% Lack of crowding/space available/adequate, enough working space.........................07 N 381 37% Comfortable desks/Chairs/Other furniture/ergonomic .................................................08 N 91 9% Plenty of filing/Storage space...........................................................................................09 N 62 6% Easy access to support/other personnel/communication..............................................10 N 96 9% Good traffic patterns/easy to move around ....................................................................11 N 79 8% Safety/security.....................................................................................................................12 N 40 4% Clean/well maintained........................................................................................................13 N 98 10% Pleasing appearance/cheery and pleasant atmosphere (color; decor; plants, etc.)...14 N 116 11% Personalized space/privacy/a space of their own to themselves.................................15 N 104 10% Ergonomic work space/work stations n ..........................................................................16 N 47 5% Adequate common space/break, lunch areas/meeting rooms/exercise room...........17 N 41 4% Up to date/proper equipment; modern technology/up dated computers; fax; phone; printers ...............................................................................................................18 N 90 9% Accessibility/availability of needed equipment/ tools (computer; fax; phone; printer)/adequate supplies ............................................................................................19 N 104 10% Location/convenience........................................................................................................20 N 19 2% Good working conditions/work environment/atmosphere(general) ..........................21 N 22 2% Air quality/ventilation........................................................................................................22 N 61 6% Layout/arrangement/organization ....................................................................................23 N 5 0% Other, non-physical attributes...........................................................................................97 N 17 2% Don't know/Not sure...........................................................................................................98 X => Q8A 92 9% Refused.................................................................................................................................99 X => Q8A 3 0%



Q7A2 N= .............................................................................................................................................. 933 100% Good lighting.........................................................................................................................1 328 35% Natural Light/Windows/Daylight.......................................................................................2 129 14% Heat/Cooling comfort/Temperature...................................................................................3 180 19% None of the above.................................................................................................................4 X 495 53%

Q8A Thinking of the factors you just mentioned as important for employee productivity, what steps have you taken to increase productivity in your workplace? N= .............................................................................................................................................. 1023 100% Nothing/No steps ................................................................................................................00 X 183 18% RECORD COMMENTS ...................................................................................................01 O 0 0% Reorganized space/work stations/remodeling/improved use/enlarged space...........02 N 363 35% Upgraded technology/equipment .....................................................................................03 N 133 13% Providing ergonomic equipment/doing ergonomic studies .........................................04 N 50 5% Better lighting/natural lighting .........................................................................................05 N 167 16% Classes/training/education.................................................................................................06 N 27 3% Purchasing new furniture ...................................................................................................07 N 67 7% Improved heating/cooling system/air quality.................................................................08 N 85 8% Listen to employees/be responsive to employees..........................................................09 N 38 4% New building/moving or moved to or leased newer/larger facility/building new facility/expanded or expanding building/landscaping building ............................10 N 75 7% Keep workspace clean, neat, well maintained/safe.......................................................11 N 66 6% Added to or improved employee amenities (break-room, concessions, restrooms, cafe, etc.) .....................................................................................................12 N 20 2% Individual/personalized workspace/privacy...................................................................13 N 40 4% Management methods/leadership/realistic goals/evaluation/holding employees accountable.....................................................................................................................14 N 5 0% Better communications......................................................................................................15 N 23 2% Improved pay scale/benefits/incentives ..........................................................................16 N 7 1% Boosting morale ..................................................................................................................17 N 26 3% Decreased workload/better workflow/less paperwork/reorganization of workload .........................................................................................................................18 N 6 1% Procedure changes(not physical workspace)..................................................................19 N 16 2% Hired more staff/employees/created new positions ......................................................20 N 9 1% Installed noise reduction measures ..................................................................................21 N 16 2% Established smaller work groups/teams ..........................................................................22 N 9 1% Implemented flex time/allow employees to have flexible schedules.........................23 N 6 1% Reduced workforce/cut costs/budgetary measures .......................................................24 N 3 0% Improved air quality/ventilation.......................................................................................25 N 13 1% Other......................................................................................................................................97 N 4 0% Don't know/Not sure...........................................................................................................98 X 42 4% Refused................................................................................................................................Q9 X 5 0%



Q9A => Q10A if Q7A2=1 AND Q7A2=2 AND Q7A2=3

I'd like to ask you about some other things that may be important to employee productivity in the workplace. N= .............................................................................................................................................. 993 100%

Q9AA => +1 if Q7A2=1

What about good lighting? Would you say good lighting is very important, somewhat important or not important to employee productivity? N= .............................................................................................................................................. 695 100% Very important ......................................................................................................................1 601 86% Somewhat important ............................................................................................................2 85 12% Not important.........................................................................................................................3 7 1% Don't know/Not sure.............................................................................................................4 2 0% Refused...................................................................................................................................5 0 0%

Q9AB => +1 if Q7A2=2

What about natural light? Would you say natural light is very important, somewhat important or not important to employee productivity? N= .............................................................................................................................................. 894 100% Very important ......................................................................................................................1 439 49% Somewhat important ............................................................................................................2 396 44% Not important.........................................................................................................................3 53 6% Don't know/Not sure.............................................................................................................4 5 1% Refused...................................................................................................................................5 1 0%

Q9AC => +1 if Q7A2=3

What about temperature? Would you say temperature is very important, somewhat important or not important to employee productivity? N= .............................................................................................................................................. 843 100% Very important ......................................................................................................................1 631 75% Somewhat important ............................................................................................................2 203 24% Not important.........................................................................................................................3 5 1% Don't know/Not sure.............................................................................................................4 4 0% Refused...................................................................................................................................5 0 0%



Q10A To further clarify your views, using the same scale, how important is it to employee productivity that employees have the ability to control the lighting in THEIR OWN workspace? IF NEEDED: Would you say this is very important, somewhat important or not important to employee productivity? N= .............................................................................................................................................. 1023 100% Very important ......................................................................................................................1 337 33% Somewhat important ............................................................................................................2 506 49% Not important.........................................................................................................................3 164 16% Don't know/Not sure.............................................................................................................4 12 1% Refused...................................................................................................................................5 4 0%

Q10B How important is it to employee productivity that employees have the ability to control the temperature of THEIR OWN workspace? IF NEEDED: Would you say this is very important, somewhat important or not important to employee productivity? N= .............................................................................................................................................. 1023 100% Very important ......................................................................................................................1 342 33% Somewhat important ............................................................................................................2 506 49% Not important.........................................................................................................................3 162 16% Don't know/Not sure.............................................................................................................4 9 1% Refused...................................................................................................................................5 4 0%



Q11 DO NOT READ PROBE TO FIT PRESS ENTER TO CONTINUE If you wanted to find out more about how to improve productivity in your workplace, where would you go? IF RESPONDENT GIVES NAME OF COMPANY, ASK: What kind of company is that? PROBE UP TO 2 MENTIONS

N= .............................................................................................................................................. 1023 100% The Internet/Worldwide Web...........................................................................................01 219 21% Betterbricks.com.................................................................................................................02 3 0% An architecture firm...........................................................................................................03 38 4% An engineering firm...........................................................................................................04 15 1% A construction firm.............................................................................................................05 11 1% A workplace consulting company....................................................................................06 148 14% An office equipment company .........................................................................................07 33 3% The library ............................................................................................................................08 64 6% Magazines ............................................................................................................................09 82 8% Use internal staff/ask employees......................................................................................10 371 36% Would attend seminars.......................................................................................................11 N 50 5% Would consult others in my industry that don't work for my company ....................12 N 71 7% Would talk to customers ....................................................................................................13 N 1 0% Would consult my particular industry's association......................................................14 N 41 4% Consult local, state, or federal regulatory agency.........................................................16 N 39 4% Newspapers..........................................................................................................................17 N 2 0% Consult the college or university......................................................................................18 N 7 1% I would rely on myself, go no where ...............................................................................19 N 10 1% I would consult my insurance carrier ..............................................................................20 N 4 0% I would consult my utility company................................................................................21 N 2 0% I would consult the phone book.......................................................................................22 N 3 0% Would keep my eyes open for mailers/ads.....................................................................23 N 3 0% Other (SPECIFY:) ..............................................................................................................97 O 2 0% Don't know/have no idea ...................................................................................................98 X 182 18% Refused.................................................................................................................................99 X 1 0%

Q12A Have you seen or heard any advertising or news stories about a website that provides information on improving productivity in commercial buildings?

N= .............................................................................................................................................. 1023 100% Yes...........................................................................................................................................1 106 10% No ............................................................................................................................................2 => Q13 897 88% Don't know/Not sure.............................................................................................................3 => Q13 20 2% Refused...................................................................................................................................4 => Q13 0 0%



Q12B PROBE TO FIT. UP TO 5 RESPONSES PRESS ENTER TO CONTINUE Where do you recall seeing or hearing the ads that mentioned improving employee productivity by improving workspaces.? PROBE: Where else? N= .............................................................................................................................................. 106 100% Conference/Business meeting...........................................................................................01 5 5% Direct Mail ...........................................................................................................................02 9 8% Internet advertising.............................................................................................................03 18 17% Magazines/Trade Journals .................................................................................................04 21 20% Newspaper............................................................................................................................05 4 4% Radio .....................................................................................................................................06 6 6% Television.............................................................................................................................07 47 44% Trade shows.........................................................................................................................08 3 3% Other (SPECIFY:) ..............................................................................................................97 O 5 5% Don't know/Can't remember.............................................................................................98 X 11 10% Refused.................................................................................................................................99 X 0 0%

Q12C What do you recall seeing or hearing in those ads?

N= .............................................................................................................................................. 107 100% RECORD COMMENTS ...................................................................................................01 O 0 0% Correct description .............................................................................................................02 N 32 30% Incorrect description...........................................................................................................03 N 17 16% Description may be correct ...............................................................................................04 N 9 8% Don't recall/Don't remember/Don't know.......................................................................98 X 49 46% Refused.................................................................................................................................99 X 0 0%



Do you recall the name of the ad sponsor or their website? 1 Yes 2 No 3 Don't know 4 Refused IF YES, ASK: What was the name (of the sponsor or website)? (If mentions BetterBricks.com, use code01, all others record under Other) 01 BetterBricks.com 97 Other (SPECIFY): 98 Don't know 99 Refused

Q12D N= .............................................................................................................................................. 106 100% Yes...........................................................................................................................................1 21 20% No ............................................................................................................................................2 80 75% Don't know.............................................................................................................................3 5 5% Refused...................................................................................................................................4 0 0%

Q12E => +1 if NOT Q12D=1

N= .............................................................................................................................................. 21 100% BetterBricks.com................................................................................................................01 11 52% Other (SPECIFY): ..............................................................................................................97 O 10 48% Don't know...........................................................................................................................98 X 0 0% Refused.................................................................................................................................99 X 0 0%

Q13 => Q14 if Q12E=01

Have you seen or heard anything about an organization or website called "BetterBricks.com"? N= .............................................................................................................................................. 1012 100% Yes...........................................................................................................................................1 95 9% No ............................................................................................................................................2 896 89% Don't know.............................................................................................................................3 21 2% Refused...................................................................................................................................4 0 0%



Q13A => Q17 if NOT Q13=1

Where do you recall seeing or hearing advertising or a story about "BetterBricks.com"? PROBE: Where else? N= .............................................................................................................................................. 95 100% Conference/Business meeting...........................................................................................01 1 1% Direct Mail ...........................................................................................................................02 3 3% Internet advertising.............................................................................................................03 7 7% Magazines/Trade Journals .................................................................................................04 9 9% Newspaper............................................................................................................................05 3 3% Radio .....................................................................................................................................06 9 9% Television.............................................................................................................................07 58 61% Trade shows.........................................................................................................................08 0 0% Other (SPECIFY:) ..............................................................................................................97 O 2 2% Don't know/Can't remember.............................................................................................98 X 12 13% Refused.................................................................................................................................99 X 0 0%

Q13B What do you recall seeing or hearing about "BetterBricks.com"?

N= .............................................................................................................................................. 95 100% RECORD COMMENTS ...................................................................................................01 O 0 0% Correct description .............................................................................................................02 N 21 22% Incorrect description...........................................................................................................03 N 8 8% Description may be correct ...............................................................................................04 N 15 16% Don't recall/Don't remember/Don't know.......................................................................98 X 51 54% Refused.................................................................................................................................99 X 0 0%

Q14 Thinking about the advertising or story you saw about "Betterbricks.com", which of the following best fits your interest in "Betterbricks.com"?

N= .............................................................................................................................................. 106 100% I am very interested in what they are saying....................................................................1 5 5% I am somewhat interested in what they are saying..........................................................2 38 36% I am not much interested in what they are saying...........................................................3 17 16% Or I don't recall enough about what they were saying to have an opinion?...............4 42 40% -------------------------------------------------........................................................................... Don't know.............................................................................................................................5 4 4% Refused...................................................................................................................................6 0 0%



Q15 Have you visited the "Betterbricks.com" website? N= .............................................................................................................................................. 106 100% Yes...........................................................................................................................................1 12 11% No ............................................................................................................................................2 94 89% Don't know.............................................................................................................................3 0 0% Refused...................................................................................................................................4 0 0%

Q15A => Q16 if Q15=1

Do you intend to visit the "Betterbricks.com" website in the next 2 weeks? N= .............................................................................................................................................. 94 100% Yes...........................................................................................................................................1 40 43% No ............................................................................................................................................2 48 51% Don't know.............................................................................................................................3 6 6% Refused...................................................................................................................................4 0 0%

Q15B => +4 if Q15A=1

How about in the next 3 months? IF NEEDED: Do you intend to visit the "Betterbricks.com" website in the next 3 months?

N= .............................................................................................................................................. 54 100% Yes...........................................................................................................................................1 14 26% No ............................................................................................................................................2 29 54% Don't know.............................................................................................................................3 11 20% Refused...................................................................................................................................4 0 0%

Q16 => Q17 if Q15B>0

Thinking about the "Betterbricks.com" website, which of the following statements best fits your response to the site . . . N= .............................................................................................................................................. 12 100% I believe what they are saying ............................................................................................1 8 67% I kind of believe what they are saying...............................................................................2 1 8% I don't believe what they are saying...................................................................................3 1 8% I didn't really understand what they are saying ...............................................................4 0 0% Or I don't recall enough about what they were saying to have an opinion?...............5 2 17% -------------------------------------------------........................................................................... Don't know.............................................................................................................................6 0 0% Refused...................................................................................................................................7 0 0%



Q16A Do you intend to visit the website again?


Q16B Do you intend to use the information from the "Betterbricks.com" website to influence the designing of the workspace for employees in your company?

N= .............................................................................................................................................. 12 100% Yes...........................................................................................................................................1 9 75% No ............................................................................................................................................2 2 17% Not my responsibility...........................................................................................................3 0 0% Don't know.............................................................................................................................4 1 8% Refused...................................................................................................................................5 0 0%

Q17 READ 1-5. ENTER ALL THAT APPLY - MULTIPLE ANSWERS OK. Which of the following types of publications do you read on a regular basis?

N= .............................................................................................................................................. 1023 100% Business weeklies, such as the Business Journal or Daily Journal of commerce....01 211 21% Local or regional newspapers (IF NEEDED: The Oregonian, Idaho Statesman, Seattle P.I.) .....................................................................................................................02 837 82% Regional business monthly magazines, such as Oregon Business or Western Business .........................................................................................................................03 213 21% Trade publications for your industry or profession.......................................................04 677 66% National Business magazines (IF NEEDED: CEO, Forbes, Business Week)..........05 267 26% ------------------------------------------------............................................................................. None of the above...............................................................................................................00 X 56 5% Don't know...........................................................................................................................98 X 1 0% Refused.................................................................................................................................99 X 1 0%

Q18 Do you have access to the Internet at home?

) N= .............................................................................................................................................. 1023 100% Yes...........................................................................................................................................1 809 79% No ............................................................................................................................................2 211 21% Don't know.............................................................................................................................3 1 0% Refused...................................................................................................................................4 2 0%



Q19 Do you have access to the Internet at work?


Q20 => +1 if Q18=2-4 AND Q19=2-4

Including both work and home, about how many total hours IN A WEEK do you usually use the Internet? N= .............................................................................................................................................. 981 100% One hour or less ................................................................................................................001 213 22% 100 hours or more .............................................................................................................100 2 0% Don't know.........................................................................................................................998 11 1% Refused...............................................................................................................................999 3 0%

Q21 If we do further research on this topic, may we contact you again? N= .............................................................................................................................................. 1023 100% Yes...........................................................................................................................................1 816 80% No ............................................................................................................................................2 194 19% Don't know.............................................................................................................................3 12 1% Refused...................................................................................................................................4 1 0%

Q21A => +2 if NOT Q21=1

Would you prefer that we contact you by phone, email or either?

N= .............................................................................................................................................. 816 100% Phone.......................................................................................................................................1 494 61% Email .......................................................................................................................................2 136 17% Either.......................................................................................................................................3 181 22% Don't know.............................................................................................................................4 4 0% Refused...................................................................................................................................5 1 0%



Q21B => +1 if NOT Q21A=2-3

What is your email address?

N= .............................................................................................................................................. 317 100% RECORD EMAIL ADDRESS.........................................................................................01 O 282 89% Don't know...........................................................................................................................02 4 1% Refused.................................................................................................................................03 2 1%

GENDR RECORD GENDER N= .............................................................................................................................................. 1023 100% Male.........................................................................................................................................1 482 47% Female .....................................................................................................................................2 541 53%

TIMEM => * if TIME/60 ; TIME DIVIDED BY 60

TIME IN MINUTES N= .............................................................................................................................................. 1023 100% ............................................................................................................................................0001 10 1% ............................................................................................................................................0002 11 1% ............................................................................................................................................0003 71 7% ............................................................................................................................................0004 200 20% ............................................................................................................................................0005 253 25% ............................................................................................................................................0006 199 19% ............................................................................................................................................0007 130 13% ............................................................................................................................................0008 62 6% ............................................................................................................................................0009 38 4% ............................................................................................................................................0010 17 2% ............................................................................................................................................0011 7 1% ............................................................................................................................................0012 7 1% ............................................................................................................................................0014 1 0% ............................................................................................................................................0015 2 0% ............................................................................................................................................0016 1 0% ............................................................................................................................................0018 1 0% ............................................................................................................................................0020 1 0% ............................................................................................................................................0022 1 0% ............................................................................................................................................0030 1 0% ............................................................................................................................................0047 1 0%

INT01 That concludes my questions. Thank you very much for your time and cooperation. Completed Interview..........................................................................................................01 1023 100%



HELP STATEMENTS FOR INTERVIEWERS * We are not selling anything. This is strictly for research purposes. * I work for Gilmore Research Group, an independent survey research firm located in Seattle. If you would like to verify this call and get more information, you may call my supervisor at 800-573-4498 ext. 176 in Seattle. * The purpose of this survey is about improving the quality of employee work space. *It's a new non-profit based in the Northwest that wants to enhance worker productivity through improved workplace design. *We have been contracted by Research Into Action to conduct the study, if you like, you may call Jane Peters at 503-287-9136 for more information. * This interview will take about 8 minutes.

F7

EBPI – MPER #1 DETHMAN & TANGORA h APPENDIX B

Appendix B: Architecture + Energy (A+E) 2001 Architecture Interview Guide

Appendix B: Architecture + Energy (A+E) 2001 Architecture Interview Guide

DETHMAN & TANGORA h EBPI – MPER #1 APPENDIX B

EBPI – MPER #1 DETHMAN & TANGORA h B - 1

ARCHITECTURE + ENERGY (A+E)

2001 ARCHITECT INTERVIEW GUIDE

Name:

Firm:

Phone number:

A+E activity:

or ___Nonparticipating member of participating firm

or ___Nonparticipant

Date of interview:

If Nonparticipant: Attempt to reach a principal or lead architect.

I am conducting research for the Northwest Energy Efficiency Alliance, a consortium of Northwest electric and gas utilities formed to address energy use in the region. The Alliance hopes to address energy use in commercial buildings and so has asked me to talk with architects about current architectural practices. Can you suggest a lead architect with your firm that I might talk with briefly?

A. First let me ask you what sectors your firm designs for.

a. Commercial facilities (such as offices, retail space, restaurants)?

Yes ___ No ___ Elaboration: __________________

b. How about government, health care, or educational facilities?


c. Industrial or warehouse facilities?


Appendix B: Architecture + Energy (A+E) 2001 Architect Interview Guide

DETHMAN & TANGORA h EBPI – MPER #1 B - 2

d. How about residential space (such as houses, apartments, assisted living)?

Yes ___ No ___

{If exclusively residential, thank and terminate}

B. Which of the following activities is your firm involved in?

a. Renovation of existing structures (if necessary: renovation is a major remodel)

Yes ___ No ___

b. Remodel of existing structures

Yes ___ No ___

c. New construction (if necessary: construction on a cleared lot)

Yes ___ No ___

d. Design/build projects

Yes ___ No ___ “A little” _______

C. About how many designers (unlicensed OK) work for your firm: ______

D. What is your role or title? ___________________{not necessary if clear its owner}

E. How long have you been practicing architecture? _____

F. Approximately how many projects did you work on last year: _______ {If 2000 participant, ask for the last 6 months}

I want to discuss with you a number of things that architects might do as part of their design practices. I would like to know the number or proportion of projects you’ve worked on in the last year/6 months that reflect these elements in their final design.

Questions relating to the earliest stages of project design:

In the last year, have you engaged in:



Y N 1. Pre-design activities to address energy and resource savings project-wide

==>If Yes, probe for specifics:

Cl Sf a. Talking to client about efficiency and resources in the pre-design stage

Cl Sf b. Setting energy efficiency goals and performance benchmarks

Cl Sf c. Educating the team on how the design affects a building’s energy use

Cl Sf d. Collaborating with consultants and contractors involved in design and construction

Cl Sf e. Other: _________________

f. # or % of projects: _____

g. Were any of these activities specifically requested by the client? {Code as Cl}

In the last year, for any of the projects was the building’s:

Cl Sf 2a. Site or orientation selected because of solar access, shading, or other resource considerations

b. # or % of projects where this was reflected in the final design: _____

c. Were any of these specifically requested by the client?

Questions relating to the building envelope and glazing:

In the last year, have you considered:

Cl Sf 3a.Designing the building envelope to reduce heating, cooling, and ventilation needs, such as through the building’s footprint, mass, skin, glazing, or tightness

b. # or % in final design: _____

c. Was this ever requested by the client?



Y N 4. In the last year, have your projects incorporated daylighting features


Cl Sf a. Lots of windows {==>this alone is not daylighting}

Cl Sf b. Incorporating shading strategies, e.g., louvers, projections, light shelves

Cl Sf c. Designing the roof to let light in, e.g., clearstories, skylights, roof monitors, stepped roofs, sawtooth roofs

Cl Sf d. Optimizing daylight penetration through location of windows in wall, floor to ceiling heights, floorplate configuration, etc.

Cl Sf e. Other: ____________

f. # or % included in final design: _____

g. Were any of these features requested by clients?

Y N 5. In the past year, have you considered passive systems to augment the electromechanical building systems


Cl Sf a. Optimizing the thermal mass of building

Cl Sf b. Using shading devices and strategies (either on the outside or inside)

Cl Sf c. Considering solar gain, night cooling, night flushing

Cl Sf d. Passive ventilation

e. Other





Questions relating to the buildings electromechanical systems:

In the last year, have you sought to:

Y N 6. Design the lighting system to be more efficient than required by code or than typically found in similar applications


Cl Sf a. Use less lights than typical; use spot or task lighting instead of general illumination

Cl Sf b. Specifying occupancy sensors or photocells

Cl Sf c. Specifying controls, switching strategies, staging sequences, stepped controls

Cl Sf d. Discuss issue with engineers

Cl Sf e. Other: ______________________



Y N 7. Design the HVAC system to be more efficient than required by code, e.g. through system selection and building design:


Cl Sf a. Designing the building to optimizing the factors that affect HVAC requirements

Cl Sf b. Comparing different types of chillers, and alternatives to chillers such as heat pumps, hydronics, radiant heat, waste heat

Cl Sf c. Considering ventilation rates, distribution systems, variable fan speeds, and variable air volume (VAV) systems

Cl Sf d. Considering controls, e.g., direct digital, integrated, user

Cl Sf e. Discuss issue with engineers



f. Use ASHRAE 90 standards (90.1 89 and 90.1 99)

Cl Sf g. Other: ______________________

h. # or % included in final design: _____

i. Were any of these features requested by clients?

Cl Sf 8a. Explored different water heating options (e.g., solar, instantaneous heating, heat recovery or reclaim)

b. # or % included in final design: _____

c. Were any of these features requested by clients?

Questions relate to methods and tools you might use:

In the last year have you:

Cl Sf 9a. Compared options by estimating life cycle cost savings from downsized equipment, reduced energy use, maintenance, and replacement cost savings

b. # or % included in projects: _____

c. Did any clients request this?

Y N 10. Used computer models to simulate building energy use or lighting


Cl Sf a. Use of models to simulate building energy use, e.g., Energy 10, DOE2, Energy Sim

Cl Sf b. Use of models to simulate lighting, e.g., daylighting models, lighting simulation modes

Cl Sf c. Other

d. # or % included in projects: _____

e. Did any clients request this?



Y N 11. Used consulting resources—either people or reference materials—assist with energy efficiency


Cl Sf a. Utility and government programs, including incentives for energy efficiency

Cl Sf b. Books, journals, websites, or CDs with methods or strategies

Cl Sf c. Consultants—project-specific or general education

Cl Sf d. Other:

e. # or % included in projects: _____

f. Did any clients request this?

Y N 12a. Are there any energy efficiency suggestions that you have made in the last year that we have not covered?


a. Other: _______________

b. # or % included in final design: _____

c. Were any of these features requested by clients?

Y N 12.2 In your experience, have clients been more concerned about energy during the last three months than they tended to be prior to that?



A+E PARTICIPANTS ONLY:

{If not A+E participant, SKIP to Q14}

13a. Because of the workshop, would you say that you suggest to clients these energy efficiency ideas more frequently, less frequently, or about the same as you did before attending any A+E events?

___ More ___ Same ___ Less

b. How about client acceptance? Would you say that your clients accept these energy efficiency ideas more frequently, less frequently, or about the same as they did before you attended any A+E events?

___ More ___ Same ___ Less

b1. {If more:} Why do you think this is so? {Probe to code:}

___ You are more persuasive, enthusiastic, knowledgeable

___ Factors external to workshop influence

c. I’d like for you to rate how valuable you found the A+E workshops to be in terms of your design work. Please use a scale of 1 to 5, where 1 signifies not at all valuable, and 5 signifies extremely valuable.

1 2 3 4 5

d. Comments about A+E:



ALL RESPONDENTS:

14. Please rate your interest in learning more about energy efficient and sustainable design practices using a scale of 1 to 5, where 1 signifies not at all interested and 5 signifies very interested.

a. 1 2 3 4 5 {If answer = 1 or 2, SKIP to Q15}

Using a 1-to-5 scale rating desirability, where 1 signifies not at all desirable and 5 signifies highly desirable, rate the following potential educational forums for learning more about energy efficient and sustainable design practices:

b. How about from journals 1 2 3 4 5

c. Books 1 2 3 4 5

d. The Internet 1 2 3 4 5

e. A teleconference/ distance learning 1 2 3 4 5

f. Workshop trainings/ seminars 1 2 3 4 5

==>If answer = 3, 4, or 5, probe for specifics:

f1. Workshop held in your office 1 2 3 4 5

f2. Workshop held in your community 1 2 3 4 5

f3. Workshop held in another city 1 2 3 4 5

f4. Workshop held in conjunction with a professional conference 1 2 3 4 5

f5. Workshop for a multidisciplinary audience: 1 2 3 4 5 i.e., gathered together are architects, engineers, lighting designers, contractors, owners, etc.



15. To what extent do the following factors limit your ability to incorporate energy efficiency features in your design work? Use a 1 to 5 scale where 1 signifies that the factor severely limits your ability to incorporate energy efficiency features and 5 signifies that the factor is not at all a limitation for you.

a. Your own interest in energy efficiency features. 1 2 3 4 5

b. Identifying energy efficient options 1 2 3 4 5

c. Assessing how well a given option will perform in a specific application 1 2 3 4 5

d. The availability of products 1 2 3 4 5

e. Getting the design team to consider energy efficient options for a project. 1 2 3 4 5

f. Getting the client to consider energy efficient options or a project. 1 2 3 4 5

g. Providing clients with reliable estimates of the costs of incorporating energy efficient features into a design. 1 2 3 4 5

h. Providing clients with reliable estimates of the benefits of incorporating energy efficient features into a design. 1 2 3 4 5

i. Identify consulting resources or other building professionals necessary to execute energy efficient design elements. 1 2 3 4 5

j. Code requirements 1 2 3 4 5

k. Other (describe) 1 2 3 4 5

{PROBING UNAIDED RECALL OF BETTERBRICKS.COM}

Y N 16a. Have you seen or heard any advertising or news stories about a website that provides information on improving employee productivity in commercial buildings? {If no, SKIP to Q17}



b. Where do you recall seeing or hearing the ads that mentioned improving employee productivity by improving workspaces? {Check all that apply.}

Y N b1. television

Y N b2. magazines/trade journals

Y N b3. newspapers

Y N b4. internet advertising

Y N b5. Other: ____________

c. What do you recall seeing or hearing in those ads? {code as:}

Y N c1. “Nailed” the description (eg., said BetterBricks.com or identified the characters or gave other specifics)

Y N c2. Answer echoed theme of question (improving employee productivity by improving workspaces)

Y N c3. Can’t recall or answer spoiled

d. Do you recall the name of the ad sponsor or their website? What is it? {code as}

Y N d1. BetterBricks.com {If yes, SKIP to Q18}

{PROBING AIDED RECALL}

Y N 17a. Have you seen or heard anything about an organization or website called BetterBricks.com? {If no, SKIP to Q19}



b. Where do you recall seeing or hearing that? {Check all that apply.}

Y N b1. television

Y N b2. magazines/trade journals

Y N b3. newspapers

Y N b4. internet advertising

Y N b5. Other: ______________

c. What do you recall seeing or hearing in those ads? {code as:}

Y N c1. “nailed” the description (eg., said BetterBricks.com or identified the characters or gave other specifics)

Y N c2. Answer echoed theme of question (improving employee productivity by improving workspaces)

Y N c3. Can’t recall or answer spoiled

IF RECOGNIZE “BetterBricks.com” NAME, ASK: {If don’t know BB.com, SKIP to Q 21}

Y N 18a. Have you visited the BetterBricks.com website? {If No, SKIP to Q19}

Y N c. Do you intend to use information from the website in your design work?

Y N d. Do you intend to visit the website again?

Y N e. Do you have any suggestions that might make the website more useful? Elaborate:

Y N 19. Do you intend to visit the BetterBricks.com website in the next 2 weeks? {If Yes, SKIP to Q21}

Y N 20. How about in the next 3 months?



ASK OF NONPARTICIPANTS:

Y N 21a. Have you heard of the Architecture + Energy Awards sponsored by the AIA, Portland chapter to recognize energy efficient building designs?

{If No, SKIP to Q22}

Y N b. Have you attended any of the award workshops? {If No, SKIP to Q22}

c. Approximately when was that? _______

==> COMPLETE Q13 ON PAGE 5

ALL RESPONDENTS

Y N 22a. Have you visited the Architecture + Energy website? {If No, Thank and Terminate. If Yes, ask:}

b. Using a 1 to 5 scale, can you rate how well the website met your expectations, where 1 signifies not at all and 5 signifies fully met or exceeded your expectations? 1 2 3 4 5

Y N c. Do you have any suggestions that might the website more useful? Elaborate:

{Thank and Terminate}



EBPI – MPER #1 DETHMAN & TANGORA h APPENDIX C

Appendix C: Northwest Energy Efficiency Alliance Efficient Building Practices Initiative – Draft Focus Group Report for the Regional Public Information

Plan’s betterbricks.com Campaign

Appendix C: Draft Focus Group Report for RPIP’s betterbricks.com Campaign

DETHMAN & TANGORA h EBPI – MPER #1 APPENDIX C


Northwest Energy Efficiency Alliance EFFICIENT BUILDING PRACTICES INITIATIVE

DRAFT FOCUS GROUP REPORT For the

Regional Public Information Plan’s betterbricks.com Campaign

Funded By:

Submitted To:

Jane Gordon, Ph.D.


Prepared By:

DETHMAN & TANGORA h




November 2000





TABLE OF CONTENTS

EXECUTIVE SUMMARY................................................................................................................C-I

INTRODUCTION AND GOALS ......................................................................................... C-I

METHODS ........................................................................................................................C-II

KEY FINDINGS AND RECOMMENDATIONS................................................................C-III Roles and Perceptions About Workspace Design Issues and Trends ......................C-III Reactions to Television Advertising ....................................................................... C-V Reactions to Print Advertising..............................................................................C-VI Characterization of the betterbricks.com Brand Identity ......................................C-VII Information Sources ............................................................................................C-VII Final Advice...................................................................................................... C-VIII

CHAPTER 1 – INTRODUCTION...................................................................................................C-1

THE EFFICIENT BUILDINGS PRACTICES INITIATIVE AND THE REGIONAL PUBLIC INFORMATION PROGRAM ..............................................................................................C-1

THE BETTERBRICKS.COM INFORMATION CAMPAIGN...............................................C-1

FOCUS GROUP GOALS ....................................................................................................C-2

FOCUS GROUP METHODS ..............................................................................................C-3

CHAPTER 2 – PERCEPTIONS ABOUT WORKSPACE DESIGN ISSUES, TRENDS, AND ROLES....C-5

MOVER AND NON -MOVER GROUPS ..............................................................................C-5 Developers/Real Estate Group ................................................................................C-7

CHAPTER 3 – REACTIONS TO TELEVISION ADVERTISING......................................................C-11

CHAPTER 4 – REACTIONS TO PRINT ADVERTISING................................................................C-15

CHAPTER 5 – THE BETTERBRICKS.COM BRAND......................................................................C-17

ADJECTIVE CHOICE......................................................................................................C-17

WHAT WILL YOU FIND AT WEBSITE? .........................................................................C-18

DESCRIPTION OF BETTERBRICKS.COM .....................................................................C-18

CHAPTER 5 – INFORMATION SOURCES AND FINAL ADVICE...............................................C-21

INFORMATION SOURCES .............................................................................................C-21

FINAL ADVICE ...............................................................................................................C-21

APPENDIX: FOCUS GROUP MATERIALS ................................................................................C-23




EBPI – MPER #1 DETHMAN & TANGORA h C - I

EXECUTIVE SUMMARY


The focus groups discussed in this report examined betterbricks.com advertising, a major communications mechanism of the Northwest Energy Efficiency Alliance’s (the Alliance) Energy Efficient Building Practices Initiative (EBPI). Through EBPI, the Alliance couples a major Regional Public Information Program (RPIP) with building code support efforts. EBPI’s aim is to increase the demand for highly energy efficient commercial buildings and enhance the capability of code-related institutions so they can meet increased consumer demand for efficient buildings. This research is part of the overall evaluation of EBPI.

While the central product for the RPIP campaign is the betterbricks.com website, the prime drivers to get people to that web site have been television and print advertising. Through its advertising contractor, Cole & Weber, the Alliance sponsored considerable up-front qualitative research with business people and supply side actors to develop the underlying logic of the advertising campaign, encompassed by the statements below.

Ø Productivity has power and is the most motivating benefit of energy efficiency for businesses

Ø Linking productivity gains to building energy efficiency improvements will increase worker demand for highly efficient commercial buildings.

Ø Using productivity as the “hook” reframes and magnifies the benefits of energy efficiency by making it a “business tool” for employees.

Ø Productivity connects easily to energy efficiency.

Cole & Weber then translated that logic into advertising media. However, as with the web site, the advertisements themselves were never tested with target audiences before being launched. Thus, as part of the evaluation of the RPIP effort under EBPI, focus groups were conducted with three target audiences to meet the following goals:


DETHMAN & TANGORA h EBPI – MPER #1 C - II

Ø Find out more about the roles of those involved in making decisions about the physical workspace and environment, and their perspectives on what is important in those decisions.

Ø Obtain feedback on the effectiveness of the TV and print advertising, including how well they understood the advertising messages, what they thought about the ads, what feelings the ads evoked, and what they thought the ads were trying to get them to do.

Ø Find out more about the sources target audiences use to get information on employee productivity and the physical workspace, and the best ways to reach them with information on those topics.


METHODS

We selected three key audiences of betterbricks.com for these focus groups

Ø Decision Makers and Influentials – because they may have immediate opportunities to decide on or influence physical workspace decisions. Since Decision-Makers and Influentials often work in tandem within an organization, we combined them into one pool, and then split this pool into two groups:

• Movers (Group 1) – Decision-Makers/Influentials who had recently moved or were in the process of moving; and

• Non-Movers (Group 2) – Decision-Makers/Influential who had not recently moved and were not contemplating a move.

Ø Developers/Real Estate Brokers (Group 3) – because they are key actors in determining the future of new commercial building design through their contacts with tenants and owners, but little research has been conducted with them for this program to date.

The focus groups were held in the Portland area in mid-October, 2000. Each group lasted approximately one and one half hour. Each group moved through a series of questions to address the goals of the focus groups listed above. In general, however, the conversation proceeded through these steps:

1. Roles and perceptions about workspace design issues and trends


EBPI – MPER #1 DETHMAN & TANGORA h C - III




5. Information sources

6. Final advice

KEY FINDINGS AND RECOMMENDATIONS

The focus group analysis revealed several major areas of insight about target audience response to and perspectives about the betterbricks.com advertising campaign and the underlying mission of the RPIP. This section summarizes the key findings, implications, and recommendations from this research. We have added quotes from focus group participants where they add value to the findings.

Roles and Perceptions About Workspace Design Issues and Trends

How Target Audiences Connect the Physical Workspace to Productivity

Key Findings: “When an employee is in a comfortable area, they work better, and especially if it’s ergonomically correct, it’s efficient for them, they produce more.” – Decision-Maker

Consistent with findings from other RPIP evaluation efforts, participants in all three focus groups realize that workspace relates to productivity. However, all three focus groups associate physical workspace productivity with topics such as layout, equipment, ergonomics, ambiance, and accessibility rather than with energy efficiency components such as lighting, air quality, and temperature.

Notably, Decision-makers and Influentials in the Non-Mover group did mention energy-related workspace components, while the Mover group did not, leading us to speculate that Movers were too involved with “space wars” to be able to think about these infrastructure changes. On the other hand, Non-Movers were thinking more about the aspects of their current work environment that needed improvements, like lighting and temperature.

Implications: The link between energy efficiency improvements and productivity is not likely to occur or rise to the top of people’s minds unless the connection is strengthened. In addition, if our speculation about Movers is correct, it will be very


DETHMAN & TANGORA h EBPI – MPER #1 C - IV

important to reach this group at the beginning of their decisions about space choices and improvements so that energy efficiency components can be integrated into their early thinking.

Recommendation: While the RPIP can take advantage of decision-makers, influential employees, and developers/real estate brokers already being concerned about productivity, the link between productivity and energy efficiency needs to be directly highlighted and justified in any informational, public relations, and advertising efforts.

Trends in New Building Design

Key Findings: “ . . .the two most important things we are asked about are comfort in the space and employee amenities.” – Senior Property Manager

“… and energy savings are big issues, because we are seeing significant increases in the cost of utility pricing (most nod in agreement).” – Real Estate Broker

Trends in building design for Class A buildings, as reported by the developer/real estate focus group participants, tended to be consistent with one another and quality oriented, including ambiance, amenities, functionality, flexibility, communications sophistication, and environmental concerns such as indoor air quality and energy efficiency. When specifically asked if they saw a trend toward greater concern with utility costs, 6 of 9 said “yes.”

Implication: Those who have their ears to the ground about the cutting edge for new commercial buildings cite trends that are consistent with RPIP concerns. This consistency can be exploited.

Recommendation: Develop targeted outreach materials, make direct contacts, and offer partnership opportunities for real estate developers and brokers based upon shared visions of the future of commercial buildings. The most effective collaborations for innovation will be with developers and brokers for Class A buildings where owners and tenants are willing to pay for quality improvements.

Key Players and Timing Issues in the Building Design Process

Key Findings: “With me, it’s our owner. He has the checkbook, so he has the greatest influence. It would also come down to targeted tenants . . . I would hope the owner sits down with the architect and tenants . . .” – Commercial Leasing Representative


EBPI – MPER #1 DETHMAN & TANGORA h C - V

“The best way for new ideas to get incorporated into building design are for the developers and tenants to work together.” – Retail Leasing Agent

The developer/real estate broker focus group identified an array of equally essential, important, and interactive players who most influence Class A commercial building design including users and tenants, architects, developers and owners, and regulating jurisdictions. Although these are the most often used terms for the players involved, people have a variety of job titles, and these may mean different things for different situations. For instance, the users could be the customers in a retail space or the workers in an office building or the owner could also be the developer.

Another important consideration mentioned by participants in all three groups is that efforts to influence energy efficiency improvements must occur early in the design process.

Implications: It will not be effective to leave any of the key players out of the RPIP strategy.

Recommendation: This finding reinforces the need to emphasize targeted communications with supply-side actors. The message and the approach will need to be tailored to each sub-group within the supply-side.

Reactions to Television Advertising

Key Findings: “It’s all style and no substance.” – Developer

“I have to admit, that’s one of my favorite ads.” – Developer “But do you know what it means?” – Group Facilitator “Oh no.” – Developer

Over half of those attending the focus groups had seen each of the two betterbricks.com TV ads before coming to the focus groups, indicating that the target groups noticed the ads. A greater proportion of the developer group had seen the ads before attending.

While there were mixed reviews about how much they liked the ads, most participants understood that the ads were related to workspace issues, and some identified they were related to productivity. About one-third of participants overall understood that the ad was asking them to visit a website (based on written individual responses; this increased to about half during the ensuing discussion).


DETHMAN & TANGORA h EBPI – MPER #1 C - VI

When asked if they understood the specific intent, services, target audiences, or sponsorship of the ads, most participants said the ads were indirect and unclear, and that they were uncertain or confused after seeing the first two 30-second spots. However, after seeing the third 10-second spot, which told and showed them the name of the web site, and told them to go to the web site to find out more about workspace improvements, they responded more positively to the overall TV campaign.

Implications: The ads were able to get the attention of target audiences but they were less effective at informing and convincing participants to go to the website.

Recommendations: If mass media TV advertising is to be continued, devise ads that take a more direct approach, including a clear explanation of the link between productivity and energy efficiency and the benefits of visiting the website. When directing viewers to the website, be sure to include both a verbal and visual instruction (i.e., voice plus displaying the address).

Reactions to Print Advertising

Key Findings: No one in the focus groups reported they had seen any of the print ads, suggesting that the impact of these ads has been minimal. However, the Decision-makers and Influentials responded much more positively to these ads, especially those in the Non-Movers group. Non-Movers liked that the ads raised really important mechanical issues that decision-makers do not normally connect to workspace planning. They also said the situations displayed in the ads were familiar problems to them. Developers were less positive about the print ads, but did acknowledge later in the discussion that the topics of lighting and heating were very important to them.

During the discussion of the print ads, several Decision-Makers and Influentials voiced concerns about the strategy of trying to motivate general employees to agitate for workspace changes. Some people thought the advertising could produce more “whining” and less work.

Implications: Print advertising does allow more detail and content to be presented to target audiences. Consistent with other findings, once the content of betterbricks.com is made clear, target audiences developed a greater appreciation for its intent, purpose, and products.

Recommendation: Print advertising should be continued, but it needs to be better placed and more frequently run.


EBPI – MPER #1 DETHMAN & TANGORA h C - VII

Characterization of the betterbricks.com Brand Identity

Key Findings: When asked to choose adjectives to describe the betterbricks.com brand, the most common types of adjectives chosen were “forward thinking” and “high tech” among Movers and Non-Movers, and “trendy” and “clueless” among the Developer/Real Estate Broker group. Although in some respects the two types of target audiences chose similar adjectives, “trendiness” among those in the Developer/Real Estate Broker group tended toward the negative, while “forward-thinking” tended toward the positive for Decision-Makers and Influentials.

After being shown a description of the betterbricks.com service, most focus group participants were completely surprised the effort was sponsored by a non-profit agency. Among Movers and Non-Movers, this was a definite positive, and they particularly appreciated the free consultation from the Help Desk advisor. Among Developers/Real Estate Brokers, however, the non-profit status was immaterial, there was little interest in a small amount of free advice, and they often felt they had other, better places to get information.

Implications: The very different reactions between supply and demand side target audiences to the attributes of the brand, to betterbricks.com’s sponsorship, and to the services provided shows the difficulty in finding an approach that will work for the individual target groups but not offend the other ones.

Recommendation: The emphasis on a “fast company” image may need to be revisited. However, other things that interest and compel both supply and demand side audiences might be substituted – such as a shared interest in achieving solutions to the real problems of lighting, air quality, and temperature. A key issue will be to evaluate how the service can be made more useful to the supply-siders, since they are likely to need even more in-depth services than those on the demand-side. Another key issue will be to promote the non-profit aspect of the service for the demand side without making it seem too slow-moving or irrelevant for the supply-side.

Information Sources

Key Findings: Both Movers and Non-Movers reported they would most often go to supply side resources – architects and engineers – when they need expertise about making workspace improvements. Some also mentioned publications and web sites. The Developer/Real Estate group also reported they would use supply-side expertise; they seemed to be especially oriented to personal relationships.


DETHMAN & TANGORA h EBPI – MPER #1 C - VIII

Implications: This finding further supports the importance of equal supply-side involvement in the RPIP.

Recommendation: As mentioned throughout this section, involve the supply side actors on an equal level with the demand side actors.

Final Advice

Key Findings: Only the Developer/Real Estate group provided some substantial “final advice,” as follows:

Ø Keep it simple and clear

Ø Coordinate your service with other related services

Ø Tell what’s in it for me

No doubt, based on other comments, many of those on the demand-side would echo these sentiments.


EBPI – MPER #1 DETHMAN & TANGORA h C - 1

CHAPTER 1 – INTRODUCTION

THE EFFICIENT BUILDINGS PRACTICES INITIATIVE AND THE REGIONAL PUBLIC INFORMATION PROGRAM

The Northwest Energy Efficiency Alliance (the Alliance) sponsors the Efficient Buildings Practices Initiative (EBPI) which couples a major Regional Public Information Program (RPIP) with a variety of building code support efforts to:

Ø Increase the demand for highly energy efficient commercial buildings.


The RPIP is the largest single component of the EBPI; its goals are to:

Ø Identify what benefits of energy-efficient building practices can motivate target audiences.

Ø Identify and address barriers to awareness and action by target audiences.

Ø Address both benefits and barriers through a coordinated marketing and information program.

Ø Overcome current marketplace inertia toward energy-efficient building practices by launching a substantial, integrated, brand based, targeted information campaign: betterbricks.com.

THE BETTERBRICKS.COM INFORMATION CAMPAIGN

The betterbricks.com campaign strategies have been to:

1. Focus upon the commercial sector only.

2. Target primarily employees and decision-makers in businesses, with a secondary focus on architects, developers, and real estate brokers, and government agencies.


DETHMAN & TANGORA h EBPI – MPER #1 C - 2

3. Develop a brand focus, taking a business point of view, centered around productivity and empowerment.

4. Develop a product – a web site that would “give them a place to go.” Traditional public information efforts would be used to “drive” target audiences to the web site.

To develop both the advertising and the web site, the Alliance hired Cole & Weber, a northwest advertising firm. As a starting point, they conducted qualitative research with business people and supply side actors to develop the underlying logic of the advertising campaign, as listed below.

Ø Productivity has power and is the most motivating benefit of energy efficiency for businesses

Ø Linking productivity gains to building energy efficiency improvements will increase worker demand for highly efficient commercial buildings.

Ø Using productivity as the “hook” reframes and magnifies the benefits of energy efficiency by making it a “business tool” for employees.

Ø Productivity connects easily to energy efficiency.

Cole & Weber translated this logic into advertising media – largely TV and print ads – designed to drive target audiences to the web site.

FOCUS GROUP GOALS

While the advertisements took advantage of extensive consumer research, they were never tested with target audiences before being launched. Thus, as part of the evaluation of the RPIP and the betterbricks.com campaign, the Alliance’s evaluation contractor (Dethman & Tangora and Research Into Action, Inc.) conducted focus groups designed to:

Ø Find out more about the roles of those involved in making decisions about the physical workspace and environment, and their perspectives on what’s important in those decisions.

Ø Obtain feedback on the effectiveness of the TV and print advertising, including how well they understood the advertising messages, what they thought and felt about the ads, and what they thought the ads were asking them to do.



Ø Find out more about the sources target audiences use to get information on employee productivity and the physical workspace, and the best ways to reach them with information on those topics.


FOCUS GROUP METHODS

Since it’s launch in May 2000, the betterbricks.com advertising campaign has directly and indirectly targeted a variety of audiences involved in deciding upon and influencing the physical workspace, including:

Demand Side Actors

• General Commercial Employees

• Business Decision Makers

• Business Influential Staff (“Influentials”)

Supply Side Actors

• Architects/Engineers

• Developers/Commercial Real Estate Brokers

• Bankers/Lenders

• Construction contractors & subcontractors

While all of these audiences are important to reach through the betterbricks.com campaign, we selected three key audiences for these focus groups:

Ø Decision Makers and Influentials – because they may have immediate opportunities to decide on or influence physical workspace decisions. Since Decision Makers and Influentials often work in tandem within an organization, we combined them into one pool, and then split this pool into the following two groups:

• Movers (Group 1, n=9) – Decision Makers/Influentials who had recently moved or were in the process of moving; and

• Non-Movers (Group 2, n=8) – Decision Makers/Influentials who had not recently moved and were not contemplating a move.

Ø Developers and (Real Estate) Brokers (Group 3, n =9) – because they are key actors in determining the future of new commercial building design through their contacts with tenants and owners, but little research



has been conducted with them for this program to date. The participants in this group tended to be involved with Class A commercial buildings (the most “upscale” building class).

The focus groups were held in the Portland area in mid-October 2000. Each group lasted approximately 1½ hours. Each group moved through a series of questions to address the goals of the focus groups listed above. In general, however, the conversation proceeded through these topics:

1. Perceptions about workspace design issues, trends, and roles in the marketplace (Developers/Brokers only)




5. Information sources and final advice

These topics make up the chapters of this report. For each area of discussion, participant insights are summarized first; then, where relevant, tables present more detailed findings. Verbatim quotes are in italics and may be edited to shorten or clarify the point being made.



CHAPTER 2 – PERCEPTIONS ABOUT WORKSPACE DESIGN ISSUES, TRENDS, AND ROLES

Focus group participants in all three groups were given sentence completion exercises to fill out on their own before introductions and the group discussion began. This exercise allowed participants to consider some of the focus group topics individually, before interacting with other participants, and to reveal their “top of mind” thoughts about relevant topics. The findings in this chapter mostly reflect the individual written comments, but points from the verbal discussion have been added if valuable.

MOVER AND NON-MOVER GROUPS

These two groups were asked to complete the following three sentences:

Ø The most important aspect of the physical workspace is . . .

Ø If my company could improve one thing about the physical workspace it would be to . . .

Ø The reason we would make major changes to the physical workspace is because . . .

Movers and Non-Movers listed many diverse concerns, often reflecting their specific situations, such as a need for more space. Movers listed more factors than Non-Movers, perhaps indicating their heightened interest in the topics, as shown in Tables C-1 to C-3. These lists show the staff involved in space decisions have many competing topics on their minds and that the priorities for these topics probably switch depending upon whether or not the business is involved with a move.

Overall, both groups related productivity concerns to the physical workspace, but this connection was much more prominent among Movers when they commented on the most important aspect of the physical workspace. On the other hand, it was more prominent with Non-Movers as a reason to move. Both sets of answers suggest productivity is a key part workspace issues and of the motivation to move. As one Mover put it:



When an employee is in a comfortable area, they work better, and especially if it’s ergonomically correct, it’s efficient for them, they produce more.

Table C-1: The Most Important Aspect of Physical Workspace Is . . .

MOVERS NON-MOVERS

• Ensure employee productivity, employee comfort and efficiency (4)

• Ergonomics, space fits function (3) – The space needs to . . . fit the needs of the person working in that space.

• Accessibility/convenient relation of space (3)

• Flexibility, adaptability of space to different needs (2)

• Can’t have people feeling like they’re w orking out of a box (2)

• Each person has own space (1)

• Pleasing to the eye – color (1)

• Adequate room to work, space for privacy (3) – Having more space – it’s just too crowded.

• Good ergonomics/physical comfort of employees (2)

• Good lighting (2)

• Accessibility, efficiency (1)

• To promote organization’s goals (1)

Table C-2: If My Company Could Improve One Thing About the Physical Workspace, It Would Be to . . .

MOVERS NON-MOVERS

• Improve aesthetics, update spaces (5) – The reception area needed to be more inviting to clients…our office looked … out dated.

• Increase overall size now/in future (2)

• More ergonomic furniture/match design to workers’ needs (2)

• More individual spaces (1)

• Do space planning during design phase (1) – Consider in the design phase issues of fit and function of the user population, rather than . . . fixing the problems (later).

• Enlarge offices, improve privacy in cubicle areas (3)

• Improve natural and artificial lighting (2)

• Improve air flow (2)

• More storage (2)

• Accessibility/convenient relation of space (1)

• Allow employees more input into decisions re: space (1)



As also pointed out by this quote, however, the elements most often associated with workspace productivity are such factors as layout, equipment, ergonomics, ambiance and accessibility, rather than the energy efficiency factors targeted by ECPI: lighting, air quality, and temperature.

Table C-3: The Reason We Would Make Major Changes to the Physical Workspace Is Because . . .

MOVERS NON-MOVERS

• To update space which is outdated stylistically or technologically (3)

• To have more space for expanding work force (3) – We have a company that’s been growing with leaps and bounds and we have an office that’s been remodeled and remodeled and added onto.

• Reorganization (2)

• Serve customer better (1)

• Increase worker productivity and efficiency by increasing comfort, accessibility (4) – Help make people’s work easier and more productive to get better results.

• Company expansion (2)

• Because we had the money (1)

• To provide more large group gathering areas (1)

Notably, however, the Non-Mover group did in a few cases mention lighting and airflow, suggesting that the Non-Mover group is dealing more with the day-to-day realities of their current workspace where these topics may be the daily fodder for complaints. On the other hand, comments by the Mover group suggest they are move involved with the exigencies of the move, such as space arrangements, adequacy of space, and updating of space.

Developers/Real Estate Group

This group completed these sentences:

Ø The most important trends in building design are. . .

Ø The people who have the most influence on building design are. . .

Ø The most likely way for new ideas to be incorporated into building design is to . . .



Their responses to these sentences are categorized in Tables C-4 to C-7 below. As shown in Table C-4, and in the quotes below, these Developers/Real Estate Professionals involved in Class A building development and leasing most often mentioned quality oriented trends in building design.

“ . . . the two most important things we are asked about are comfort in the space and employee amenities.” – Senior Property Manager

“ . . . and energy savings are big issues, because we are seeing significant increases in the cost of utility pricing (most nod in

agreement).” – Real Estate Broker

Table C-4: The Most Important Trends In Building Design Are . . .

DEVELOPERS / REAL ESTATE PROFESSIONALS

• Sense of ambiance, comfort, amenities, entertainment value for users (n=6)

• Functionality for users/tenants, including functional linkages between related work groups and between buildings (e.g., mixed use) (n=3)

• Flexible work spaces, such as non-stationery walls to provide maximum flexibility to tenants (n=3)

• Communications and telecom services (n=3)

• Mixed use, village or town center concept (n=2)

• Environment (IAQ) and energy savings (n=2)

• Other items mentioned once each: appearance, government regs, space per person, ease and price of construction]

• Note: There was also a substantial discussion of true value engineering and oriented buildings: “Whether you build to hold or you are a merchant developer where you build it with the idea of selling it, the idea is still the same. The nicer the building you build, the more value you create, and of course the higher sales value . . .the idea of value engineering.”

Key trends included a focus on ambiance, amenities, functionality, flexibility, communications sophistication, and, notably for EBPI, environmental concerns such as indoor air quality and energy efficiency. Value engineering – “to create a very high value building with integrating experience both in the construction field, the design field, and the tenant use field” – was also discussed.

While only one person brought up energy efficiency, when all were specifically asked if they saw a trend toward greater concern with utility costs, 6 of 9 said “yes.” The



trends listed in this group also echo the concerns mentioned in the Decision-maker/Influential groups.

The Developer/Real Estate group identified an array of equally important players who interact to influence the design and development of commercial space. As shown in Table C-5, these key players include users and tenants, architects, developers and owners, and regulating jurisdictions (noted as getting more and more important). As the table also suggests, the terms “user” and “tenants” can change depending on the building type, and a developer could also be an owner, but not necessarily. The next quote points out that multiple players need to be kept in mind if influence is to be achieved.

“With me, it’s our owner. He has the checkbook, so he has the greatest influence. It would also come down to targeted tenants . . . I would

hope the owner sits down with the architect and tenants . . .” – Commercial Leasing Representative

Table C-5: People Who Have the Most Influence on Building Design Are . . .


• Users, tenants (n=6) (customers for retail spaces and are employees in offices)

• Architects (n=6)

• Developers or owners (n=5)

• Jurisdictions who regulate building construction, aesthetics, placement, etc. (n=4)

• The person paying the bill – could be tenants or developers (n=2)

• Other – one mention: contractors, construction coordinator, spouses, brokers, the market

Table C-6 points out that it is a combination of “who” brings in the new ideas and “when” they are brought in. Those who develop and lease buildings say their most important sources of new ideas are both on the demand and supply side, and that the time to get in is early, in the planning and “building the building” stage, as pointed out below.

“ The best way for new ideas to get incorporated into building design are for the developers and tenants to work together. Those are the two

people who are going to get the job done.” – Retail Leasing Agent



“There’s building the building and there’s building out the inside of the building. There’s two pieces to this puzzle.“ – Developer

Table C-6: The Most Likely Way for New Ideas to Be Incorporated Into Building Design Is to . . .


• Developers/owners learn of these trends through communication with users/tenants, as well as with architects, contractors (n=5)

• Discussion must occur early in the design phase between relevant players (named elsewhere) – point here is timing (n=4)

• An experienced developer knows how to adapt to changes in the market, and must be flexible and accommodate new trends (n= 3)

• Employees give feedback to landlords or facilities managers and this in turn is communicated to developers (n=2)

• Employees give feedback to landlords or facilities managers and this in turn is communicated to developers (n=2)



CHAPTER 3 – REACTIONS TO TELEVISION ADVERTISING

Each of the groups watched all three television ads for betterbricks.com, but not all in the same order, to avoid any ordering effects. After seeing their advertisement, the focus group participants filled out a “thought bubble” which asked them to characterize the ad’s message, what they thought about the ad, what their feelings were about the ad, and what they thought the ad wanted them to do. For subsequent advertisements, they were asked to add anything new to their ideas.

Movers and Developers watched the 30-second Consultant spot first. This spot satirizes the “omnipotent” management consultant, who is only paying attention to the people aspects of work (when he’s not paying attention to himself). The consultant’s sidekick points out that the physical workspace could also be important, much to the consternation of the consultant. The betterbricks.com web site then is shown on the screen.

Non-Movers watched the 30-second Cheerleader spot first (Sparky). This ad features an office cheerleader, Sparky, trying to enliven the troops by her ebullient spirit, despite some difficulties in the physical office environment. Once again, the ad points out that the physical workspace may also be important to work getting done. Again, the betterbricks.com website is posted on the screen.

Both groups watched a 10-second spot third. This spot used a voice over to present the connection between the physical workspace and productivity and to urge the viewer to go to the betterbricks.com website, which is also shown on the screen.

Over half of the 26 focus group participants remembered seeing each of the 30-second spots before coming to the group (15 of 26 for the Consultant and 18 of 26 for the Cheerleader). The Developer/Real Estate group was most likely to have seen them, and Sparky was the most memorable. No one had seen the 10-second spot.

While there were mixed reviews about how much they liked the ads, most participants understood that the ads were related to workspace issues, and some identified they were related to productivity. About one-third of participants overall understood that the ad was asking them to visit a website (based on written individual responses; this increased to about half during the ensuing discussion). Thus, the ads were able to get the attention of many in the target audiences and to communicate they were about workspace issue; however, the ads appear to be less effective at directing audiences to the website.



When asked if they understood the specific intent, services, target audiences, or sponsorship of the ads, most participants said the ads were indirect and unclear, and that they felt uncertain or confused after seeing the first two 30-second spots. Many found the Consultant ad hard to follow, although they said it got clearer at the end:

“It was very fast paced; I found myself trying to catch up with what was going on.” – Mover

However, several said it was clearer than the Cheerleader spot. Quite a few found the Consultant insulting and “arrogant,” especially among the Developer/Real Estate group.

Reaction to the Sparky ad was generally more positive across the groups, with people especially appreciating the sense of humor, fewer finding the main character annoying, and several mentioning that it was a little slower paced and easier to follow. Still, the quotes below summarize reactions to the 30-second spots:

“It’s all style and no substance.” – Developer

Then about the Sparky ad: “I have to admit, that’s one of my favorite ads.” – Developer

“But do you know what it means?” – Group Facilitator “Oh no.” – Developer

After seeing the third 10-second spot, which told and showed them the name of the web site, and told them to go to the web site to find out more about workspace improvements, they responded more positively to the overall TV campaign. All three groups said the ad was direct, straightforward and told them what to do:

“It actually tells you to go to betterbricks.com; that stuck with me better and made me want to go to the web site.”

They liked the double approach of seeing and hearing the web site address.

While several people liked being entertained and appreciated the entertainment value of the 1st two ads, the majority in both groups said the direct approach which told them where to go (the website) and what they would learn was more effective and helpful.

“The intent [of the first 2 ads] sometimes got lost. You like them, laugh at them. But the last commercial I came away knowing more what the message is.”



Several people also pointed out the value of the combination of all the ads – that the entertaining ads get your attention, and the direct ad tells you what to do. They also recognized that a single 10-second ad would not be enough to familiarize them with betterbricks.com.





CHAPTER 4 – REACTIONS TO PRINT ADVERTISING

Focus group participants were then presented the print advertisements for betterbricks.com. The full color print ads had been run in some regional business journals and other publications and focused upon specific physical workspace issues such as poor lighting and ventilation. Participants were asked to give their first impressions and then to highlight in the ads what they felt was positive and memorable and what was not.

No one in the focus groups reported they had seen any of the print ads, suggesting that the impact of these ads has been minimal. However, the Decision-makers and Influentials responded much more positively to these ads than the Developers and Real Estate Professionals, especially those in the Non-Movers group.

Both Movers and Non-Movers liked that the ads raised really important mechanical issues that decision-makers don’t normally connect to workspace planning. They also said the situations displayed in the ads were familiar problems to them, and they liked the humor with which they were presented. They also appreciated that these ads allowed more detail and content about productivity and the workspace concerns they targeted. Their single complaint was the use of an “armpit” statement.

“These ads tend to focus on things you don’t usually think about in the workspace. We usually think about the physical space, computers, cubicles, etc. But this other stuff (temperature, air, light) is just as

important – it’s the environment.” – Mover

“ Lighting, temperature and air flow are all critical – I hear about these a lot.” – Non-Mover “I listen to 90% of employees whining daily about

temperature.” – Mover

When asked who the ads were aimed at, there was no real consensus among the Mover group, but the Non-Movers said office users and decision-makers about space and noted that this was the same as for the TV ads. A few people suggested that the ads could encourage employees to “whine.”

Developers and Real Estate Professionals were less positive about the print ads, but did acknowledge later in the discussion that the topics of lighting and heating were very important to them. This group commented that they:



Ø Were confused about the name betterbricks.com

Ø Thought company behind betterbricks.com was a space planner or a general contractor or an HVAC co. like Carrier

Ø Thought the ads were aimed at space planners and architects

Ø Expect to see such ads in trade magazines



CHAPTER 5 – THE BETTERBRICKS.COM BRAND

ADJECTIVE CHOICE

When asked to choose three adjectives from a page of adjectives that would best describe the betterbricks.com brand, the most common types of adjectives chosen were “contemporary,” “forward thinking” and “high tech” among Movers and Non-Movers, and “forward-thinking,” “trendy” and “clueless/inferior” among the Developer/Real Estate Broker group, as shown in Table C-7 below.

Table C-7: Adjectives Chosen for the betterbricks.com Brand

MOVERS NON-MOVERS DEVELOPERS

• Contemporary/progressive/ alternative/trendy/forward-thinking = 9

• Exciting/noticeable/unique/energetic/pro-active = 9

• Caring/friendly/ approachable = 4

• Humorous = 3

• Contemporary/progressive/ trendy/forward-thinking = 8

• Straightforward/high tech/ practical/expert/efficient/ high quality/confident = 7

• Personal/approachable/good natured/amicable = 5

• Humorous = 3

• Noticeable/pro -active = 2

• Contemporary/ alternative/ trendy =4 / forward-thinking = 8

• Clueless (n=5)/ condescending/ aggressive/inferior = 9

• Free-spirited/adventurous = 3

• Personal/approachable = 2

• Humorous = 2

Although in some respects the two types of target audiences chose similar adjectives, “trendiness” can be a double-edged attribute. Those in the Developer/ Real Estate group tended more toward the negative in subsequent discussion of their adjective choices, especially when the “clueless” comments are figured in. Decision-Makers and Influentials tended to be more positive, emphasizing the “forward-thinking” aspects, but still wondered how long this “trend” would last, as the quotes below show:

“This is a new trend… we’re becoming more aware of how these issues, ergonomics, air, light, affect productivity; it may not last.” – Mover



“In that they go about it in real different ways.” – Non-Mover

The Developer/Real Estate group discussed their adjective choices quite extensively, saying the ads were very unclear, and emphasizing clueless, but often in a kindly manner, as shown in the quote below.

“I don’t think the ads are really all that trendy. They might be clueless in their advertising, but that doesn’t demonstrate a cluelessness as a

company. I think they just might be a little confused.” – Developer/Real Estate Group

This “kindly” tone also related to a number of participants in the supply-side group emphasizing that the ads were forward-thinking, even if somewhat confused at times. These remarks were along the lines that heating and lighting really are important, and if this is what the ads and betterbricks.com were really about, they’d like to know more.

“I was most affected by the print media where the HVAC system and lighting were focused on. These are forward-thinking people. They’re saying, ‘don’t just settle for what’s here. Think about what could be.’”

WHAT WILL YOU FIND AT WEBSITE?

Only the Decision-Makers and Influentials were specifically asked this question, although the Developers and Real Estate Professionals often hazarded guesses throughout the discussion. Several people said that if they visited the website, they would find be a consulting firm – “Some way to try to snag you so you’ll want to follow-up and spend some money” – to help improve the workspace, showing some offices (before and after analysis), and some links to other sites for equipment or space manufacturers.

DESCRIPTION OF BETTERBRICKS.COM

After being shown a description of the betterbricks.com service, most focus group participants were completely surprised the effort was sponsored by a non-profit agency. Among Movers and Non-Movers, this was a definite positive, as these quotes show:

“If I knew this I may be more inclined to go to the website because I’m not getting hooked into something where they’ll squeeze money out of me. Rather, this won’t hurt; I’m not obligated to anything.” – Mover



“These [air, temp, lights] are three issues we are facing in our workspace and it’s going to take renovation and some funding… but this gives me hope that I could go to my administrator and say here’s

something we could research and it wouldn’t be so costly.” – Non-Mover

The demand side groups particularly appreciated the free consultation from the Help Desk advisor. However, the issue of “The amount of money they put into marketing… where’s that money coming from?” did come up.

Among Developers/Real Estate Professionals, however, the non-profit status was immaterial; there was little interest in a small amount of free advice for the large size projects they handled (although the amount of free time was misrepresented in the description as one hour, not four hours, it’s doubtful this would have changed their reactions significantly); they often felt they had other, better places to get information (such as utilities for energy efficiency advice); and they were still confused about betterbricks.com.

The very different reactions between supply and demand side target audiences to the attributes of the brand, to betterbricks.com’s sponsorship, and to the services provided shows the difficulty in finding an approach that will work for the individual target groups but not offend the other ones.





CHAPTER 5 – INFORMATION SOURCES AND FINAL ADVICE

The final discussions in the focus groups asked participants where they would go to get information about the topics that were being discussed, and if they had final advice for betterbricks.com sponsors. The amount of time spent on these topics varied greatly.

INFORMATION SOURCES

Both Movers and Non-Movers reported they would most often go to supply side resources and mostly to people – architects and engineers and ergonomic experts – when they need expertise about making workspace improvements. Some also mentioned publications, web sites, and trade shows, related to their specific businesses or trades (e.g., schools), but not general publications, sites, or shows.

A few also mentioned they would go to colleagues doing the same type of work they did – for instance, facilities directors, and that they might visit facilities they liked to find out who designed them and what worked, didn't work. While web sites were not top-of-mind to most, when prompted, they did say they would go to the Internet for information, but not only to the Internet.

When asked about who might be a credible spokesperson for betterbricks.com, they found that hard to identify, but suggested “any CEO of a large company that had actually used it (betterbricks.com).”

The Developer/Real Estate group also reported they would use supply-side expertise; they seemed to be especially oriented to personal relationships, saying “Give me a name and phone number.” They mentioned a number of professional association meetings that could also be helpful, such as monthly BOMA meetings, and agreed that builder and trade shows could be very helpful.

FINAL ADVICE

Only the Developer/Real Estate group provided some substantial “final advice,” as follows:

Ø Keep it simple and clear



Ø Coordinate your service with other related services

Ø Tell what’s in it for me

And, in terms of a final word on an ad that would affect them, one developer said this was the ultimate “What’s in it for me ad.”

“If in the first ad was a guy sitting on a beach in the Bahamas, and another guy asked him how he afforded the trip, the guy would say: ‘I saved my compnay such-and-such amount of dollars, and they gave me a bonus and sent me on a trip because I called betterbricks.com. There you go.”



APPENDIX: FOCUS GROUP MATERIALS





BETTERBRICKS.COM

BUSINESS INFLUENTIALS / DECISION-MAKERS

FOCUS GROUP DISCUSSION GUIDE

FINAL

1. 10/11/00 6:00 pm – 7:30 pm (90 minutes) – Recent/current/future change in workspace

2. 10/11/00 8:00 pm – 9:30 pm (90 minutes) – No change in workspace

3. 10/12/00 6:00 pm – 7:30 pm (90 minutes) – Developers/Real Estate Professionals

5 min. I. INTRODUCTION

Ø Purpose of the Group (for participants) – Learn how key people in organizations think about improving their workspace

Ø Focus group set up (recording, one-way mirror, observers)

Ø Rules of participation

Ø Agenda overview

10 min. II. THOUGHT BUBBLES (Provide Participants with 1st HANDOUT:”)

Ø Individual warm-up exercise – Take a few minutes to write your responses to these “thought bubbles.” (complete the sentences)

− The most important aspect of the physical workspace is . . .

− If my company could improve one thing about our physical workspace, it would be to. . .

− The reason we would make major changes to the physical workspace is because . . .



Ø Go around the room, have people introduce themselves and what their job is, then tell their thought bubbles. Write on three flip charts

[subtotal=15 minutes]

20 min. III. TV ADVERTISEMENTS

Ø Ad exposure exercise – I’m going to show you a TV ad that’s been running recently during the Olympics. Some of you may remember it. Pass out and explain 2nd Handout (see below). Use C&W’s basic thought bubble that has “message, think, and feel” bubbles, and have participants add “do” bubble after they see the ad.

Ø After you see the ad, I’d like you to fill in the thought bubbles on this sheet and then we’ll discuss what you wrote. I’d like you to put in:

− The message the ad is conveying, wants to leave you with

− What you think about the ad – what’s going through your mind as you watch

− What feelings or emotions the ad evoked

Ø Then show one BB.com ad.

Ø NOW, add a 4th bubble:

− What the ad wanted you to do

Ø First, how many of you remember seeing this ad before? (count)

Ø What was the message the ad wanted you to take away with you?

Ø What kind of organization sponsored the ad?

Ø Probe on any confusions if apparent.

Ø Probe on productivity and energy issues if mentioned.

Ø What did you think about the ad as you watched it?



Ø What feelings did the ad evoke?

Ø What that ad was trying to get you to do, if anything. (see website probes below)

Ø Now I’m going to show you another ad and I’d like you to note on the bottom of your thought bubbles (by 2nd ad notes) any new reactions or information you got from the ad. (Show the second ad.) Let’s discuss your new or added reactions and information.

Ø Here’s a 10-second ad that is meant to work with the other ads, so assume you are seeing it in conjunction with the other ads. As you watch it, write your thoughts on your thought bubble sheet next to “10 second spot.” (Show 10 second spot) Does this ad give you any other thoughts and insights about the message and purpose of the ads ?

Ø Now I’d like you to tell me what you think about the company/brand suggested in the advertising. Here’s an adjective checklist ( 3rd Handout); please circle the three adjectives that best describe the company/brand in this advertising. Discuss choices and reasons.

[subtotal 35 minutes]

20 min. IV. PRINT ADVERTISEMENTS

Ø Now, here are the print ads for the same campaign we’ve been discussing. (Show three ads on boards.)

Ø Has anyone seen these ads before coming here tonight? Where?

Ø After a first glance at the three ads, please tell me:

Ø What messages do the ads convey? Are they all conveying the same message?

Ø Who are the ads aimed at?

Ø What associations do you make with the ads? What feelings do they evoke?



Ø What do these ads want you to do if anything? Do they all want you to do the same thing?

Ø Where would you expect to see ads like these?

Ø Print ad exercise #2. Pass out copies of the ads, 4th Handout, to everyone and a yellow and a blue highlighter. As you read through the copy of each ad, I’d like you to highlight in YELLOW any words or phrases that attracted your interest, and highlight in BLUE any words or phrases that your didn’t find interesting or that might have confused you.

Ø Now, let’s take the first ad. After reading the copy, what do you think the ad is about?

Ø Who is the ad aimed at?

Ø What interested you, didn’t interest you, and why? (yellow highlights)

Ø What didn’t interest you or confused you and why? (blue highlights)

Ø Did the ad motivate you to do anything? What? Why or why not?

Ø Did you have any other important reactions to the ad?

Ø Follow same pattern for the remaining two ads.

Ø Now let’s talk about all three ads together, now that you’ve looked at them more closely.

Ø Are the ads conveying similar messages?

Ø Did you believe the ads? Trust them?

Ø Did they seem important? Why or why not?

Ø Who do you think is sponsoring the ads? Profit? Non-Profit? Does it make a difference?

Ø Now after seeing all the ads for this campaign, have your impressions of the brand changed? How?



Ø Now, some of you said that the ad was trying to get you to visit a website. (If no one noticed, say “No one has mentioned this, but the ad included a website that the sponsors hoped you would visit.) If you visited that website, what would you expect to find there? Why would you expect to find those things at the website? How interested would you be in visiting that type of website? Why or why not?

Ø Present "What is betterbricks.com? "Is this consistent with what you know so far? Any surprises? Reactions to the brand description?


20 min. V. SOURCES OF INFORMATION – We’ve been talking about improving the physical environment of workspaces. Let’s say your company has decided to either make a major renovation to your present workspace or move to a new workspace. You’ve been put in charge of making sure the physical workspace enhances employee productivity.

Ø How important would you feel this assignment was? Why or why not?

Ø To what types of publications or other sources of information do you regularly go to keep track of the business environment?

Ø Where would you go to get information on employee productivity and workspace? Why? (Brainstorm and record. If not mentioned, probe the following:

− The Internet/Worldwide Web

− Architecture firm

− Engineering firm

− Construction firm

− Real estate professional

− A workplace consulting company



− An office equipment company

− The library

− Magazines, newsletters -- specific ones?

− Internal staff

− Professional organizations/colleagues elsewhere

− Newspapers, business newspapers

Ø If the organization sponsoring the effort we’ve been discussing tonight wanted to get you to pay attention to information about employee productivity and workspace environment, where should they advertise? Publish news or magazine stories? Are there other ways they should get the word out?

Ø Who would be credible spokespeople?

Ø What kinds of information would you be looking for or would be most valuable?


10 min. VI. MESSAGES – On this page (5 th Handout) are three messages that betterbricks.com is trying to get across to business decision-makers. I’d like you to read the first one, and tell me your reactions to it.

Ø Probes: Does the statement seem true? Does it resonate with your experience? How would you improve the message?

− Companies spend an enormous amount of time addressing daily workplace annoyances: thermostat wars, computer screen glare caused by improper lighting, distracting noises and smells passing through poorly insulated walls and windows.

− Entrepreneurs will discover that a small change in workplace energy practices--improved lighting, temperature and air quality--yields a big payoff in employee productivity. And improved productivity means an improved bottom line.



− Betterbricks.com is a new online resource arming businesses with straightforward building solutions to improve workplace comfort, efficiency and productivity. In an age where businesses routinely scrutinize and squeeze employee profitability, betterbricks.com has uncovered a competitive edge by tapping revenues from an unsuspected source: the workplace itself.

[subtotal=85]

5 min. VII. WRAP UP – We’ve talked about a lot of things here tonight. What I want to do now is get your final pieces of advice for betterbricks.com. Betterbricks.com’s “bottom line” is this (show big board with these statements):

− A more healthy and comfortable employee is a more productive employee.

− Productivity can be improved through better temperature control, better air quality, better lighting, and better noise control.

Ø What final advice would you give betterbricks.com for advancing these ideas and for encouraging employers to put these elements in their workspace?

[total=90]

THANKS VERY MUCH!





BETTERBRICKS.COM

REAL ESTATE DEVELOPERS / PROFESSIONALS

FOCUS GROUP DISCUSSION GUIDE

FINAL

1. 10/11/00 6:00 pm – 7:30 pm (90 minutes) – Recent/current/future change in workspace

2. 10/11/00 8:00 pm – 9:30 pm (90 minutes) – No change in workspace

3. 10/12/00 6:00 pm – 7:30 pm (90 minutes) – Developers/Real Estate Professionals

5 min. I. INTRODUCTION

Ø Purpose of the Group (for participants) – Learn perspective about how to foster improvements in the physical workspace

Ø Focus group set up (recording, one-way mirror, observers)

Ø Rules of participation

Ø Agenda overview

15 min. II. THOUGHT BUBBLES (Provide Participants with 1 st HANDOUT:”)

Ø Individual warm-up exercise – Take a few minutes to write your responses to these “thought bubbles.”

− The most important trends in building design are . . .

− The people who have the most influence on building design are . . .

− The most likely way for new ideas to be incorporated into building design is to. . .



Ø Go around the room, have people introduce themselves and tell what their job is (NOTE: Spend time on understanding their jobs.) Then have them tell their thought bubbles. Write on flip chart.

Ø Can you tell me more about the roles each of you have in bringing a project to the market? Your relationships with each other?

Ø Can you tell me more about how you determine the needs of the market and how that affects what you build?


15 min. III. TV ADVERTISEMENTS Ad exposure exercise – I’m going to show you a TV ad that’s been running recently during the Olympics. Some of you may remember it. Pass out and explain 2nd Handout (see below). Use C&W’s thought bubble that has “message, think, and feel” bubbles, and have participants add “do” bubble after they see the ad.

Ø After you see the ad, I’d like you to fill in the thought bubbles on this sheet and then we’ll discuss what you wrote. I’d like you to put in:

− The message the ad is conveying, wants to leave you with

− What you think about the ad – what’s going through your mind as you watch

− What feelings or emotions the ad evoked

Ø Then show one BB.com ad.

Ø NOW, add a 4th bubble:

− What the ad wanted you to do

Ø First, how many of you remember seeing this ad before? (count)

Ø What was the message the ad wanted you to take away with you?

− What kind of organization sponsored the ad?



− Who was the ad aimed at? (If not them, why not? If the same information were aimed at them, how would it be different?)

• Probe on any confusions if apparent.

• Probe on productivity and energy issues if mentioned.

Ø What did you think about the ad as you watched it?

Ø What feelings did the ad evoke?

Ø What did the ad want to viewer to do, if anything?

Ø Now I’m going to show you another related ad and I’d like you to note on the bottom of your thought bubbles (by 2nd ) any new reactions or information you got from the ad. (Show the second ad.) Let’s discuss your new or added reactions and information.

Ø Here’s a 10-second ad that is meant to work with the other ads, so assume you are seeing it in conjunction with the other ads. As you watch it, write your thoughts on your thought bubble sheet next to “10 second spot.” (Show 10 second spot) Does this ad give you any other thoughts and insights about the message and purpose of the ads?

[subtotal 35 minutes]

20 min. IV. PRINT ADVERTISEMENTS

Ø Now, here are three print ads for the same campaign we’ve been discussing. (Pass out ads.)

Ø Has anyone seen these ads before coming here tonight? Where?

Ø After a first glance at the three ads, please tell me:

− What messages do the ads convey? Are they all conveying the same message?

− Who are the ads aimed at? (If not them, why not? If the same information were aimed at them, how would it be different?)



− What associations do you make with the ads? Are you interested in the ads? What feelings do they evoke?

− What do these ads want you to do if anything? Do they all want you to do the same thing?

− Where would you expect to see ads like these?

PERHAPS LEAVE OUT: Print ad exercise #2. Pass out copies of the ads, 4th Handout, to everyone and a yellow and a blue highlighter. As you read through the copy of each ad, I’d like you to highlight in YELLOW any words or phrases that attracted your interest, and highlight in Pink any words or phrases that your didn’t find interesting or that might have confused you.

Ø Now let’s talk about all three ads together, now that you’ve looked at them more closely.

− Maybe leave out: Discuss ads together – interested, not interested, other reactions – why?

− Are the ads conveying similar messages?

− Did you believe the ads? Trust them?

− Did they seem important? Why or why not?

− Who do you think is sponsoring the ads? Profit? Non-Profit? Does it make a difference?

− Now after seeing all the ads for this campaign, have your impressions of the brand changed? How?

Ø Now I’d like you to tell me what you think about the company/brand suggested in the advertising. Here’s an adjective checklist ( 3rd Handout); please circle the three adjectives that best describe the company/brand in this advertising. Discuss choices and reasons.

Ø Now, some of you said that the ad was trying to get you to visit a website. (If no one noticed, say “No one has mentioned this, but the ad included a website that the sponsors hoped you would visit.) If you visited that website, what would you expect to find there? Why



would you expect to find those things at the website? How interested would you be in visiting that type of website? Why or why not?

Ø Present "What is betterbricks.com? "Is this consistent with what you know so far? Any surprises? Reactions to the brand description? Do you agree with bb.com’s premise?


20 min. V. MARKETPLACE DEMAND AND SOURCES OF INFORMATION

Ø Betterbricks.com wants to get high quality work environments into commercial buildings, through things like improved lighting, daylighting, temperature control etc.

Ø When you’re developing or marketing space, and you’re talking with potential tenants about what they’re looking for, how concerned are they about what makes a work environment high quality or more productive? Is high quality lighting, temperature etc important? How do those discussions go? What do you tell them?

Ø If not asked about these things, what are tenants mostly concerned with? If developers/real estate folks were asked about these things, what kind of difference would it make to their building or marketing decisions?

Ø If betterbricks.com wants developers and real estate professionals to build and market buildings with higher quality, more productive work environments, what needs to happen? How to they get your interest and support? What information or messages or services or support would get your attention?

Ø Now I’d like to know more about where you get information.

− First, how to you regularly keep track of trends and developments that affect your business? People? Publications? Other means?

− What methods to you use to get information to the marketplace about your buildings? (Probe with list below)



• The Internet/Worldwide Web

• Architecture firm

• Engineering firm

• Construction firm

• Real estate professional

• A workplace consulting company

• An office equipment company

• The library

• Magazines, newsletters -- specific ones?

• Internal staff

• Professional organizations/colleagues elsewhere

• Newspapers, business newspapers

Ø Who would be credible spokespeople about building design?

Ø What are the most effective ways to reach you with new information?

Ø What kinds of information would you be looking for or would be most valuable?

(subtotal=75 minutes]

15 min. VI. MESSAGES AND WRAP-UP On this page (6 th Handout) are three messages that betterbricks.com is trying to convey. I’d like you to read the first one, and tell me your reactions to it.

Ø Probes: Does the statement seem true? Does it resonate with your experience? Does it apply to your line of work? How? How would you make them more convincing or relevant?



− Companies spend an enormous amount of time addressing daily workplace annoyances: thermostat wars, computer screen glare caused by improper lighting, distracting noises and smells passing through poorly insulated walls and windows.

− Entrepreneurs will discover that a small change in workplace energy practices--improved lighting, temperature and air quality--yields a big payoff in employee productivity. And improved productivity means an improved bottom line.

− Betterbricks.com is a new online resource arming businesses with straightforward building solutions to improve workplace comfort, efficiency and productivity. In an age where businesses routinely scrutinize and squeeze employee profitability, betterbricks.com has uncovered a competitive edge by tapping revenues from an unsuspected source: the workplace itself.

Ø What advice would you give betterbricks.com for advancing these ideas in the marketplace and with you?

[total=90]

THANKS VERY MUCH!



EBPI – MPER #1 DETHMAN & TANGORA h APPENDIX D

Appendix D: Northwest Energy Efficiency Alliance Efficient Building Practices Initiative – Draft betterbricks.com Website Usability Report

Appendix D: Draft betterbricks.com Website Usability Report

DETHMAN & TANGORA h EBPI – MPER #1 APPENDIX D


Northwest Energy Efficiency Alliance EFFICIENT BUILDING PRACTICES INITIATIVE

DRAFT BETTERBRICKS.COM WEBSITE USABILITY REPORT

Funded By:

Submitted To:

Jane Gordon, Ph.D. Northwest Energy Efficiency Alliance

Prepared By:

Jerrold Prothero, Ph.D. Duff Hendrickson

Rita Solon Jaan Sunderlin

Hypercerulean, Inc.


Research Into Action, Inc.

DETHMAN & TANGORA h Linda Dethman

Dethman & Tangora, LLC

September 2000 Revised November 2000





TABLE OF CONTENTS

EXECUTIVE SUMMARY................................................................................................................ D-I

INTRODUCTION AND GOALS .........................................................................................D-I

METHODS .........................................................................................................................D-I

KEY FINDINGS................................................................................................................D-II

CHAPTER ONE – INTRODUCTION, PURPOSE, AND METHODS ................................................ D-1

METHODS ........................................................................................................................ D-1

ORGANIZATION OF THE REPORT ................................................................................. D-3

CHAPTER TWO – USABILITY ISSUES AND RECOMMENDATIONS............................................. D-5

COMMUNICATION OF WEB SITE PURPOSE ................................................................ D-5

ORGANIZATIONAL ISSUES ............................................................................................ D-5

NAVIGATIONAL ISSUES ................................................................................................ D-7 Discover, Believe, Act Tabs .................................................................................... D-7 Home Button ......................................................................................................... D-7

CONTENT ISSUES........................................................................................................... D-8

STYLISTIC ISSUES.......................................................................................................... D-8

Font Size and Contrast .......................................................................................... D-8 Do Not Assume Fast Links or High-End Computers .............................................. D-8 Use Left Margin of Home Page for Navigation, Not a Picture ................................ D-9

ADVISOR AREA ISSUES ................................................................................................. D-9 Poor Placement ..................................................................................................... D-9 Length and Detail ................................................................................................D-10 Lack of Clarity About Results ...............................................................................D-10

FIELD STUDY ISSUES ...................................................................................................D-10 Difficulty in Finding the List of Field Studies.......................................................D-10 Field Study Options are Not Found ......................................................................D-11 360 Degree View Functionality Unclear................................................................D-11 Cannot Smoothly Exit 360 Degree View Tool ........................................................D-11 Field Study Window Can Not Be Resized..............................................................D-11

CHAPTER THREE – RESULTS OF TASKS AND ANSWERS TO QUESTIONS................................. D-13



APPENDIX A: TABLE OF USABILITY PROBLEMS....................................................................... D-21

APPENDIX B: PARTICIPANT COMMENTS............................................................................... D-31

APPENDIX C: USABILITY TEST PROCEDURES.......................................................................... D-37

USABILITY TEST PROCEDURES ..................................................................................D-39 Background..........................................................................................................D-39 Participant Introduction .......................................................................................D-39

PHASE ONE: FIRST IMPRESSION ................................................................................D-40

PHASE TWO: TASKS ......................................................................................................D-40 Task Two: Extended Exploration ..........................................................................D-40 Task Three: Gathering Decision -Related Information ...........................................D-41 Task Four: Test Specific Features or Areas...........................................................D-41

PHASE THREE: POST -TEST QUESTION .......................................................................D-41

APPENDIX D: BACKGROUND ON HUMAN COMPUTER INTERFACES.................................. D-43 I: Why HCI Matters ................................................................................................D-43 II: General Interface Design Principles....................................................................D-44 III: Menu Organization ...........................................................................................D-45 IV: Short-Term Memory .........................................................................................D-45 V: Response-Time Delays .....................................................................................D-46 VI: The Use of Color .............................................................................................D-46


EBPI – MPER #1 DETHMAN & TANGORA h D - I

EXECUTIVE SUMMARY


This usability test examined the betterbricks.com web site, a product of the Northwest Energy Efficiency Alliance’s (Alliance) Energy Efficient Building Practices Initiative (EBPI). Through EBPI, the Alliance couples a major Regional Public Information Program (RPIP) with building code support efforts. EBPI’s aim is to increase the demand for highly energy efficient commercial buildings and enhance the capability of code-related institutions so they can meet increased consumer demand for efficient buildings. This research is part of the overall evaluation of EBPI.

The betterbricks.com website is the RPIP’s campaign’s central information product to inform businesses about the worker benefits of high efficiency choices in commercial buildings. When the website was developed, usability testing with potential users was not conducted. Usability testing, which is usually qualitative, examines how visitors to a site actually behave when using a web site and focuses on their requirements.

After the launch of the website a number of users noted enough difficulties and “bugs” in the site that the Alliance decided to sponsor usability testing to look for:

Ø Misconceptions or confusions about the purpose of the site

Ø Task-performance and navigational problems

Ø Items or components that visitors particularly liked or disliked.

METHODS

Hypercerulean, Inc, a Seattle firm that specializes in web site usability studies, along with Alliance evaluation staff and EBPI’s overall evaluation contractors, Dethman & Tangora and Research Into Action, Inc, designed the test procedures. Hypercerulean conducted the tests in two locations – Seattle, Washington and Boise, Idaho – in late August 2000. Participants were recruited to match various target audiences of or potential visitors to the web site: general commercial employees; influential staff who strongly influence space decisions; decision makers


DETHMAN & TANGORA h EBPI – MPER #1 D - II

who make the final decisions about space; architects; and developers. In all, 17 people, from the web site’s target audiences, participated in one and one half hour in-depth individual interviews. The results reflect the web site at the time of the testing.

KEY FINDINGS

While over 100 usability-related comments and actions were recorded for betterbricks.com, the key issues were poor communication of the purpose and organization of the web site. Participants were frustrated by not knowing what the web site was supposed to do; what information was available; and where to find it. As participants became more familiar with the site most came to like it more (12 of 17 participants) and 11 of 17 said they would recommend the site to co-workers, managers, or clients.

The participants generally thought the content was useful, once they found their way around. However, as the site evolves participant feedback suggests that further information should be provided in many areas, such as how to carry out changes; how to evaluate problems in detail; before and after studies; regulatory issues; where to reach appropriate contractors; etc.

Six significant categories of usability problems emerged from participant testing:

1. Unclear Purpose: From the TV ads, participants tended to think that betterbricks has something to do with office furniture, or possibly Internet services. While they came to understand that the web site has something to do with the quality of the workspace, none were very clear on the purpose of the web site or of Betterbricks.com. This lack of clarity tended to make them want to leave the web site, and makes them less willing to recommend it to others.

3. Poor Web Site Organization: None of the participants had a clear idea of where to start; what range of tools and information was available; or how the parts of the web site were connected to each other. This caused considerable frustration.

4. Poor Navigational Tools: The Discover/Believe/Act tabs tended to be rarely used, for reasons discussed in Section 3.0. Suggested improvements are given there as well.

5. Further Content Is Needed: As the site evolves, Section 3.0 discusses further content information that participants will need.


EBPI – MPER #1 DETHMAN & TANGORA h D – III

6. Problems With Advisor: In principle, the offer of free expert advice should be a major draw for Betterbricks.com. As currently implemented, it is instead a major weakness actually driving visitors away. See Section 3.0 for a discussion and recommendations.

7. Problems With Field Studies. The field studies are very popular (although more detailed information is desired by many participants). Operational difficulties currently limit their usefulness, however. See Section 3.0 for a discussion and recommendations.

These six problem categories, and how to address them, are discussed in more detail in Chapter Two.


DETHMAN & TANGORA h EBPI – MPER #1 D - IV


EBPI – MPER #1 DETHMAN & TANGORA h D – 1

CHAPTER ONE – INTRODUCTION, PURPOSE, AND METHODS

This usability test examined the betterbricks.com website, a product of the Northwest Energy Efficiency Alliance’s (the Alliance) Energy Efficient Building Practices Initiative (EBPI). Through EBPI, the Alliance couples a major Regional Public Information Program (RPIP) with building code support efforts to:

Ø Increase the demand for highly energy efficient commercial buildings.


The overall goal of evaluating EBPI, of which this usability test is a part, is to assess how well and to what extent this dual approach works to change markets.

The betterbricks.com web site is the RPIP’s campaign’s central information product to inform businesses about the worker benefits of high efficiency choices in commercial buildings. When the web site was developed, usability testing with potential users was not conducted, primarily due to the need to launch the site simultaneously with the media campaign. Usability testing, which is usually qualitative, examines how visitors to a site actually behave when using a web site and focuses on their requirements.

After the launch of the web site however, a number of users noted enough difficulties and “bugs” in the site that the Alliance decided to sponsor usability testing to look for:

Ø Misconceptions or confusions about the purpose of the site

Ø Task-performance and navigational problems

Ø Items or components that visitors particularly liked or disliked

METHODS

Hypercerulean, Inc, a Seattle firm that specializes in web site usability studies, along with Alliance evaluation staff and contractors, designed the test procedures. Hypercerulean conducted the tests in two locations – Seattle, Washington and Boise, Idaho – in late August 2000. Participants were recruited to match various


DETHMAN & TANGORA h EBPI – MPER #1 D - 2

target audiences of or potential visitors to the web site: general commercial employees; influential staff who strongly influence space decisions; decision makers who make the final decisions about space; architects; and developers. In all, 17 people, from the web site’s target audiences, participated in one and one half hour in-depth individual interviews, as shown in Table D-1 below.

Table D-1: Distribution of Test Participants by Location & Target Audience

PARTICIPANTS SEATTLE BOISE TOTAL

General Commercial Employees 2 2 4

Influential Employees 3 2 5

Decision Makers 2 1 3

Architects 2 1 3

Developers 1 1 2

Total 10 7 17

The testing proceeded in three phases:

Ø Phase 1 – First Impressions: After seeing two betterbricks.com TV ads, participants were asked to behave as if they were visiting the site by themselves in real life. They explored the site, unguided, for about five minutes and described aloud to the interviewer what they were doing and thinking.

Ø Phase 2, Tasks: Participants further explored the site, initially without guidance, and were asked to continue to report their behaviors and reactions. Then, after being shown relevant betterbricks.com print ads, they were asked to gather information from the site to convince colleagues of the importance of the physical work environment. Finally, participants were asked to visit specific site locations or features (if they already hadn’t) and asked for their reactions.

Ø Phase 3, Follow-Up Questions: After spending an hour at the site, participants were asked follow-up questions about a range of web site impressions and issues.



ORGANIZATION OF THE REPORT

After the Executive Summary and this Introduction, Chapter Three organizes the results of the usability test in terms of the key problems found and recommendations for resolving them.

Chapter Four provides outcomes of participant tasks performed on the web site and answers to questions raised by the betterbricks team prior to the usability test.

Appendix A provides a table of 135 potential usability issues in the parts of www.Betterbricks.com tested, along with their frequency of occurrence among the test participants.

Appendix B provides comments by participants on various issues.

Appendix C provides general background information on human-computer interfaces.





CHAPTER TWO – USABILITY ISSUES AND RECOMMENDATIONS

This section organizes the results of the usability test in terms of the key problems found and recommendations for resolving them.

COMMUNICATION OF WEB SITE PURPOSE

Issue: While all of the participants realized that the BetterBricks site has to do with improving the work environment, none were certain precisely how, nor what services Betterbricks.com provides. This was a major source of frustration for participants. It contributed both to a desire to leave the web site initially, and to the unwillingness of a third of the participants to recommend the site to others. Web visitors want to know quickly and clearly why a web site exists.

After about an hour on the web site, 10 of 17 participants thought Betterbricks.com was a for-profit organization, 5 thought it was a not-for-profit, and 2 were unsure. Six of the participants indicated that they would prefer for BetterBricks to be a not-for-profit organization, because of trust and competitive issues. None indicated that they would prefer for BetterBricks to be a for-profit organization.

Recommendation: Provide a mission statement prominently on the home page. State that BetterBricks is a not-for-profit and why it was created. List any endorsements from professional or trade organizations. State who the web site is intended for. In the words of one participant, tell visitors “what inspired the company to exist.”

ORGANIZATIONAL ISSUES

Issue: Participants did not know where they should go first, or what paths to follow through the site. The division into “Discover”, “Believe,” “Act” meant little to them and did not provide a natural division of the web site’s information. Most of the participants did not find the list of all field studies. Nine of 17 participants did not find their way to the Act area without prompting.

Web visitors tend to be in a hurry: if they don’t get oriented quickly, they leave. They want to know what their options are on a site, and how to get to them. Ease of access to information also affects their willingness to recommend a site to others.



Recommendation: Organize the web site in terms of tasks the visitors want to perform and tools for performing those tasks. For instance, a menu on the home page might have entries such as:

Ø Learn more about:

• Betterbricks

• Lighting

• Heating

• Air quality

• Affect of work environment on health

• Affect of work environment on performance

• Field studies at corporate and government sites

− Select kinds of field studies you want:

§ Large/Medium/Small Company

§ Urban/Rural Location

§ <Climate/Region Options>

Ø Research Studies

Ø Tool to rate your work space

Ø What you can do for your work space

Ø Tools to convince others

Ø Get free advice on your work space

Ø Companies that can help you improve your work space

Ø Regulatory issues for work spaces

Ø Related sites

Ø Privacy statement



The key point is, lay the home page out clearly in terms of tasks visitors want to perform. This will allow them to quickly orient themselves, and to see value that will make them want to stay and to recommend the site to others. Provide layers of detail, so that visitors start at a general overview level and can drill down to details as needed.

Do not be afraid to have a boring design. Web users for professional sites are much more frustrated by difficulty accessing information than by lack of artistic originality. The telephone book is not in every home and business because of the quality of its visual design.

NAVIGATIONAL ISSUES

Discover, Believe, Act Tabs Issue: Participants rarely tried to use these tabs and had considerable difficulties when they did try. There were at least four problems: 1. They are somewhat difficult to notice, being above and disconnected from the main body of the web site; 2. The meaning of the terms “Believe”, “Discover” and “Act” was not obvious to participants, which did not inspire them to click on them; 3. The relationship between the menus and the submenus was unclear to participants; and 4. The menu-submenu structure was very difficult to use. Unless the mouse is moved very precisely, the submenus change while the participant is in the process of navigating to them from the main menu. This is very confusing, as well as requiring considerable time and attention.

Recommendation: Get rid of the entire structure. Replace it with a left margin navigational bar. See the section on Organization Issues above for further suggestions.

Home Button

Issue: None of the 17 participants were familiar with the convention that clicking on the logo in the upper left corner takes one to the home page. They all took significantly longer than necessary to return to the home page, by repeatedly clicking the “Back” button. This is an important problem because the home page is the center of operations. Web visitors need to be able to get to it quickly.

Recommendation: Include a 'Home' button on each web page.



CONTENT ISSUES

Issue: Participants generally liked the content on the BetterBricks web site, once they got past problems of organization and navigation. However, the content is mostly introductory, and did not give them the information they would need to take action. Issues not fully addressed included: what companies can they contract to for good workspace support? How can they evaluation their lighting, HVAC, etc.? What can they do short-term? What can small companies do? What are the cost, timeline, and personnel requirements? What are the health, productivity, and financial issues, under various assumptions? Are these kinds of data available for the field studies? Is there good “before and after” information available? What are the regulatory issues? Are there research studies available on these topics?

Recommendation: As the web site evolves, integrate in more detailed information. Provide a simple “suggestions” e-mail link to gather information about what visitors want to know more about. Be careful to follow a clean organizational structure, as described above. As more detailed information is added it should not obscure the high-level overview.

STYLISTIC ISSUES

No major stylistic problems were found in this usability test. A few points that did come up are listed below.

Font Size and Contrast

Issue: Five participants mentioned that the fonts were too small in parts of the web site. Also, in places light lettering is used on a light background. Both of these conditions discourage reading of the text.

Recommendation: Be aware of these issues. Provide good contrast for all text. Remember that developers are often younger, and have better monitors, than the target audience. Test the web site informally with older viewers on lower-end monitors.

Do Not Assume Fast Links or High-End Computers

Issue: While the BetterBricks web site is targeted primarily at professionals likely to have high-speed links and high-end computers at work, they are not necessarily



viewing the web site from work. Particularly if their attention was captured by a TV ad, they are likely to be at home.

One participant in this study is responsible for space management and purchasing for a research lab of a major manufacturing company. He mentioned that about 50% of the time, he visits web sites from at home, over a 28K modem with a small monitor. He also mentioned that since companies often loan aging computers to employees for home use, many employees have older systems at home.

Recommendation: Keep graphics to a minimum and test the web site on low-end systems. Consider removing image fade-ins, which (as one participant put it) have lost their “wow” value.

Use Left Margin of Home Page for Navigation, Not a Picture

Issue: Web visitors need to be given a very quick sense of what the site is about and how it can help them. The decision to stay or leave an informational site such as www.Betterbricks.com will be driven much more by the usefulness of the content than by the artwork. For web sites in general, it is not at all uncommon for a visitor to decide to leave before the home page has finished loading.

Recommendation: Use critical space on the home page to plainly express the purpose of the site and what tools it provides. Use the top for a mission statement and the left margin for a navigational bar (see also the discussion of organization, above). Graphics should be secondary to communicating purpose and capabilities.

ADVISOR AREA ISSUES

The possibility of receiving free, personalized advice is in principle a major draw for BetterBricks. In practice, however, it received only a lukewarm reception from participants. The reasons for this are discussed below.

Poor Placement

Issue: In the current design, participants frequently reached the Advisor area as their first stop. This raised at least two problems: 1. web visitors are very reluctant to fill out forms before they have visited a site and are convinced of its value. Seeing a form on their first link tends to make them want to leave the site. 2. The form was occasionally mistaken as a registration step, necessary before visiting the rest of the site.



Recommendation: Make certain that the Advisor area is not the first place visitors are drawn to. See also the discussion of organization, in Section 3.2.

Length and Detail

Issue: Due to limited time, the target audience for Betterbricks.com is reluctant to fill out a long form as a means of initial contact. Also, one participant mentioned that he would need BetterBricks to sign a non-disclosure agreement before his company would allow him to answer some of the questions.

Recommendation: Make first contact less intimidating. Consider allowing visitors to simply enter their contact information and free form text. More detailed information can be requested as needed for a subsequent round of communication.

Lack of Clarity About Results

Issue: Participants had two concerns about the result of filling out the form: 1. what useful information they would get back; and 2. whether they would be spammed. (The text at the top of the Advisor section tends to get missed, since it is in smaller font and the eye is directed to the form below.)

Recommendation: Provide a prominent privacy statement. State prominently at the top of the form how the visitor will benefit from filling it out.

FIELD STUDY ISSUES

The field studies were generally liked, although there was a desire for more detail about techniques used, benefits, before vs. after comparisons, costs, timelines, personnel requirements, contractors used, etc. Participants were also unclear about what role, if any, BetterBricks played in the field studies. The 360-degree view option was very popular, once it was found and its operation understood.

The below discusses a few operational problems.

Difficulty in Finding the List of Field Studies

Issue: While some of the field studies were found through the examples, the list under a submenu from the tabs at the top of the page was rarely found without prompting.



Recommendation: Make it more obvious both that there is a collection of field studies and how to reach them. See the discussion of organization and navigation in Sections 3.2 and 3.3, above.

Field Study Options are Not Found

Issue: Eight of 17 participants did not notice the viewpoint options at the top of the field studies. The 360-degree view option was similarly not noticed by many participants. Reasons for this probably include their small size and that attention is not drawn to them. One participant also commented that there were so many buttons that he just ignored all of them.

Recommendation: Look for ways to make these options more prominent, as they were popular once pointed out. One option is simply to increase their size. Another would be to mention there existence at the bottom of the text for the field study.

360 Degree View Functionality Unclear

Issue: It was not immediately clear to any of the participants how to operate the 360-degree view tool. Most expected it to either show a movie or a single, different viewpoint. When neither happened, their instinct was to click on the “download” link. They did not tend to read the (somewhat lengthy) instructions.

Recommendation: Making the 360-degree view tool easier to use is important, since it is very popular once understood. Simplify the text instructions to a statement such as the following, in large font: “Please put the mouse over the picture and click to move viewpoint.”

Cannot Smoothly Exit 360 Degree View Tool

Issue: Once the 360-degree view option is selected, there does not appear to be a way to return to the previous state.

Recommendation: Support a clean return from the 360-degree view tool.

Field Study Window Can Not Be Resized

Issue: Clicking on the “expand” button on the upper right corner of the window does not change its size.



Recommendation: Support window resizing for the field studies.



CHAPTER THREE – RESULTS OF TASKS AND ANSWERS TO QUESTIONS

This chapter gives the results of the usability test from the point-of-view of tasks & questions developed prior to the experiment. In the below table, “(Phase 1)”, “(Phase 2)”, “(Phase 3)” indicates when in the test the particular information was gathered. Numbers in parentheses after a statement refer to the number of participants for which the statement was true.

Table D-2: Results of Usability Test

BETTERBRICKS TEST OBJECTIVES USABILITY TEST PARTICIPANT ANSWERS

1.) (Phase 1) Was the site what they expected from the TV ads they saw?

Yes (7); no (4); didn’t know what to expect from the ad (4); not sure yet (2). Since the TV ads were very general and none of the participants developed a precise idea of what the Betterbricks.com web site was for, it is hard to say how good the match actually was. Most participants thought from the TV ads that BetterBricks had something to do with office furniture or office layout. A few thought it had to do with Internet or computer services.

2.) (Phase 1) What were their first thoughts about the web site?

What’s the product? If it’s trying to sell something, I haven’t got the pitch yet (4);

It’s about improving the work space (2);

Like the layout, lettering size, not crowded (2);

Felt like leaving. Where am I supposed to go? (2);

Like that it doesn’t have flashy ads (1);

Like that it’s “cut and dry” (1);

Don’t know yet (1);

Colors are good (1);

Colors are bad (1);

Has to do with well-being, quality of life issues (1);

Want to learn more (1);

Who are they, and who are they targeting? (1);

Doesn’t do much for me (1);

Flashy, lot of work, but not well organized (1).

Continued




3.) (Phase 1) What first drew their attention?

The picture on the left side (11);

Jazzed (1);

“Create” (1);

“Does your space inspire?” (1);

Large text on right (1);

Pictures on the far right column (1); Looked for what company about (would leave because didn’t find it) (1).

4.) (Phase 1) Where do they think they would spend the most time on the site?

Don’t know (6);

Looking for solid info (3);

Building Loyalty (2);

Great Spaces (2);

Trying to find where to go, what the options are (1);

Better Bricks, productivity-related information (1);

Healthy Habitat and air quality information (1);

First temperature and light info, then productivity (1).

5.) (Phase 1) How do they describe the purpose of the site, after initial exploration?

To create a better work environment (6);

To help managers make decisions (2);

Selling something, not clear what (2);

No idea (2);

Consultant clearinghouse (1);

Commercial for healthy office environments (1);

Reference information on various topics (1);

Possibly selling planning or evaluation services (1);

Shows need to be proactive about workspace (1).

6.) (Phase 1) Who do they think this site is intended for?

Management (12);

Both management and workers (1);

Separate parts different target audiences (1);

Management, HR, people in transportation, logistics, shipping, etc. (1); Commercial developers, interior designers, furniture suppliers, maybe employees (1);

People who would buy or specify products (1).

Continued




7.) (Phase 1) Would they continue exploring the site after the first few minutes? Would they come back for a second visit?

Yes (10); No (7). (The number of “yes” responses is probably inflated somewhat by the test procedure. In real life, some of the yeses would probably have left immediately, before Phase 1 was completed and the question asked. After spending a few minutes on the site, participants tended to find i t more interesting.)

8.) (Phase 1) Did they find out anything new during the first few minutes?

No (13); Yes (4). The “Yes’” tended to have to do with a better, but incomplete, understanding of what BetterBricks is about. “They don’t sell office equipment.” – P5.

9.) (Phase 1) Did they think the site prompted or encouraged them to do anything?

No (10); To try to figure out what it’s about (3); to continue exploring (2); not sure (1). None indicated that it motivated them to address workspace problems.

10.) (Phase 2) Did their impressions of the site change after a longer exploration?

No (9); like it more (7). “At first, I thought it was an advice line. Now, I think it’s a consulting service” (1). Generally, the impression of the site improved over time. Many participants found valuable information, although almost all were frustrated by organizational problems and by not knowing what BetterBricks does.

11.) (Phase 2) What did they think about the “Rate Your Space” Questionnaire?

It was generally thought to be a useful diagnostic tool.

12.) (Phase 2) What did they think about the Advisor area?

The participants in this test, like web users in general, were very reluctant to fill out a long form, particularly giving out contact information. The conditio ns under which they said they would fill it out tended to be:

If they already understood BetterBricks and its services; Knew what to expect as a result of filling out the questionnaire (including whether they would get spammed or not); and Had a specific problem in mind which they were pretty sure BetterBricks could help with.

To get a higher rate of response, consider: Putting the questionnaire at the end of a step-by-step introduction to BetterBricks, its service, and work space issues in general; Providing a privacy statement, including what sort of mailings they are likely to get, if any; Shortening it considerably, possibly with follow-on questions after the first contact.

Continued




13.) (Phase 2) What did they think about the field studies?

Most liked the field studies, particularly the 360-degree views. Eight participants did not notice the top bar of the field study window, which contained iconic links to other views. About the same number did not notice the 360-degree view button. “There were too many buttons, so I ignored them all,” was one comment. All of the participants were initially confused about how to operate the 360-degree view tool. Most expected it to either show a motion picture or else simply a different view. Several clicked (unnecessarily) on the “download” link in an attempt to get something to happen. Problems noted by the participants included the lack of hard information on costs, timelines, before vs. after comparisons, etc. Who should they contact for similar services? There was confusion about how BetterBricks was related to the field studies.

14.) (Phase 3) Had their impressions of the site changed by the end of the test?

No (5); slightly better (4); better (7); much better (1). Participants tended to rate the site more highly as they found more information and learned to overcome navigational difficulties. None of the participants rated the site worse after a longer exposure to it.

15.) (Phase 3) Did they find out anything new? If so, what?

Yes (12); No (5). Examples of what was learned centered around the importance of air, light and temperature issues and that a site existed that integrated information about them.

16.) (Phase 3) Would they do anything new as a result of visiting the site?

No (7); Yes. Tell people about it (5); Yes. Use it to help with space planning (2). “Does this have anything to help small companies doing a remodel?” –P15

17.) (Phase 3) How would they describe the site?

How to improve workplace condition, through light, air and heating (5);

A reference site on workspace information (2);

Examples, specialized problems, etc. about work spaces (2);

Difficult to find information (2);

Not sure yet (2); A broad overview, with case studies, of environmentally friendly systems (1);

Shows why some work sites better than others (1);

Great site, but ignore the first page (1);

Awkward, confusing, not to the point (1);

Very user-friendly. Not hard to understand (1).

Continued




18.) (Phase 3) Would they recommend the site?

Yes (11); No (6). The “No”es generally had to do with uncertainty about the purpose of the site and difficulty getting information out of it.

19.) (Phase 3) What do they like about the site?

The field studies (3);

Informative (3);

Not full of ads (3);

360 degree view interesting (3);

Colors (2);

Nothing (2);

Printable posters (1);

Lots of places to explore (1);

Useful now for space planning (1);

Cares about environment & health (1);

Not trying to sell, no membership restrictions (1);

The home page, but should point to ‘Better Bricks’ first (1);

The potential (1);

Not ‘over vivid’ (1).

20.) (Phase 3) What don’t they like about the site?

Poor navigation (5);

Don’t know what it is about (3);

Small fonts (3);

Too hard to use (2);

Wordy (2);

Colors weak, poor contrast (2);

Don’t know who they are (2);

Not direct (1);

White letters (1);

Poor contrast for text (1);

Colors in Act (1);

Don’t know what to do first (1);

Scroll bar (1);

Tabs at top (1);

Lack of privacy statement (1);

Too general and repetitive. Not enough information (1);

No people in the pictures, so seem like sterile places (1).

Continued




21.) (Phase 3) What would they change or add?

Make it more clear (4);

Say who BetterBricks is (2);

Needs “About Us” on the home page (2);

Add more details on products or services (1);

Get rid of Java animations (1); Remove blurry picture, provide menu of about company, what BetterBricks does, etc. (1);

Don’t know (1);

Have a serious commercial (1);

Emphasize field studies, 360 deg. (1);

“Act” options sooner. Too long get there (1);

Provide cross-references, summaries, bulleted lists (1);

The home page has to say what you do (1);

More productivity statistics (1);

Guidance on how to change own layout (1);

Make ‘BetterBricks’ first thing you get to (1);

Provide more specific information (1);

Reduce the amount of text (1);

Provide hard information (1);

Remove white text on light colors (1); Add cost information, by region. E.g., there are snow problems in North Dakota (1);

Want to be able to plot grid of office space, get ideal layout (1); “For the field studies, provide specific costs, timeline, how many people involved, etc. Cost information up front does not discourage real customers. Provide more depth on the process, as well as results.” – P16.

22.) (Phase 3) Would they like to see more detailed information?

No (7);

Yes. Studies on products (6);

Yes, if clearly presented (2);

Not sure (1). The “No” responses were partly a result of not feeling a need for further detail. A second concern was that more detail would increase the navigational problems.

23.) (Phase 3) Would they like to see more field studies?

Yes (10); No (5); Only if they have before and after comparisons on productivity, effect on employees (2).

“How about field studies for small companies with less money?” –P15

Continued




24.) (Phase 3) Would they like more information on how to carry out changes?

Yes (11); No (3); Provide this information through links to other sites (1).

25.) (Phase 3) Would they like more information about financial benefits?

Yes (14); No (1); If linked to field studies (1).

The “No” was skeptical of financial benefits information in general.

26.) (Phase 3) Do they think the information on the web site is accurate, complete and unbiased?

Accurate: yes (11); no (1); not sure (5).

Complete: yes (5); no (6); not sure (5).

Unbiased: yes (9); no (6); not sure (2). For those who thought it biased, one concern was that there was no discussion of both pros and cons (3); another was that it might all be a sales pitch (2).

27.) (Phase 3) Why do they think BetterBricks is providing the web site?

To promote BetterBricks sales of products or services (9); Don’t know (7); as a portal to businesses (1).

28.) (Phase 3) Do they think BetterBricks is for-profit or not-for-profit? Does it matter to them?

For profit (10); not-for-profit (5); don’t know (2).

Doesn’t matter which it is (11);

Prefer if not-for-profit (6). None expressed a preference for BetterBricks to be a for-profit organization. It was felt that information from a non-profit would be less biased and more reliable. Also, one architect pointed out that if BetterBricks was a for-profit, it would be a potential competitor of his.

29.) (Phase 3) Would they like the site to contain more information about BetterBricks?

Yes (16); No (1). Participants wanted this information clearly available on the home page. Lack of understanding of who BetterBricks was a major frustration for participants in trying to understand the web site, and a significant factor for those who said they would not recommend the site to others.

Continued




30.) (Phase 3) How well did they think the TV and print ads introduced the site?

Good (10); Poorly: thought site was about office furniture or computer/Internet services (5);

Poorly: TV ads were annoying (1). The TV ads were generally thought to be funny, but did not convey a good sense of what BetterBricks is about. The leading guess from the TV ads was that BetterBricks sold office furniture. There was some feeling that they would be more likely to look the site up if it were more informative. Several of the participants mentioned that when they had seen the ads at home on television they had not realized that BetterBricks had a web site. “Betterbricks.com ” comes up rather softly while the attention is focused on the actors’ expressions. One participant suggested a series of TV ads showing a progression of BetterBricks-related improvements. The ads could still be funny, but would be more informative. Eventually, we would find out what Sparky’s real job was, because she would be able to get back to it.

The print ads were generally thought to be more informative, but less amusing and still not clear about what BetterBricks does.

31.) (Phase 3) What other comments did they have?

See Appendix B, below, for a list of participant comments. The web site was generally praised for content, although there was a desire for more specifics (e.g., what were the costs and timeline for the field studies, what were the before and after productivity statistics, what equipment is appropriate in different circumstances, etc.).

The web site was heavily criticized for poor organization and lack of clarity about who BetterBricks is and “what inspired it to exist”.



APPENDIX A: TABLE OF USABILITY PROBLEMS

Comprehensive list of usability-related issues from the usability test. Some of the frequencies are approximate.

Table D-3: Table of Usability Problems

PROBLEM ID PROBLEM DESCRIPTION FREQUENCY OF OCCURRENCE (N = 17)

1 Poor organization of home page. 13

2 BetterBricks pitch unclear 8

3 Want more background information before contacting BB in Advisor.

8

4 Boxes of text on home page are not links. 1

5 Lack of solid information quickly. 7

6 Would come back after initial visit. Yes (10); No (7).

7 Who is the web site aimed at? Management (12);

Both management and workers (1);

Separate parts different audiences (1); Management, HR, people in transportation, logistics, shipping, etc. (1); Commercial developers, interior designers, furniture suppliers, maybe employees (1); People who would buy or specify products (1).

8 Has problems with tabs at top. 17. (Most did not use at all. When prompted to, all had initial difficulty with tab to sub-tab movement.)

9 What was BB’s role in the field studies? A few.

10 Who should I hire if I like the service? 1

Continued




11 Doesn’t like the picture fade-ins in the field studies: takes time, no added value.

1

12 Audio would disturb co -workers. 2

13 Dislikes pop-ups, which also cause navigational problems.

2

14 Looks like an online ergonomics textbook. 1

15 Thinks this is a portal for an association of companies.

2

16 Why so few links to other sites? 1

17 Wants solid research references. 1

18 Didn’t get to “Act” area without explicit prompting.

9

19 Doesn’t want to give BB a lot of private information early on.

2

20 Can’t read the text on the e -postcards in the Act area, and would therefore not use them.

1

21 “Rate your space” questions too vague to answer.

1

22 Likes the 360-degree facility. All or almost all.

23 Too much fluff. 2

24 Needs to get to information faster. 1

25 Learned something new. Yes (12); No (5).

26 Would recommend? Yes (11); No (6).

27 Likes printable posters. A few.

28 Get rid of moving visual effects (fade-ins, not 360).

1

29 Get rid of blurry picture on home page. 1

Continued




30 Want menu on left side of home page. With “About the company,” “What they do”, “What action I can take”.

1

31 Want before & after comparisons for field studies.

1

32 Thinks accurate. Yes (11); No (1); Not sure (5).

33 Thinks complete. Yes (5); No (6); Not sure (5).

34 Thinks unbiased. Yes (9); No (6); Not sure (2).

35 Thinks not-for-profit. For profit (10); Not-for-profit (5); Don’t know (2).

Doesn’t matter which (11);

Prefer not-for-profit (6).

36 How well did ads introduce site? Good (10); Poorly: thought site was about office furniture or computer/Internet services (5);

Poorly: TV ads were annoying (1).

37 Discover, belief, act are not intuitive. 1

38 Doesn’t understand or use logo = link to home page, when asked to return to home page from several pages down.

All.

39 First drew attention: The picture on the left side (11);

Jazzed (1);

“Create” (1);

“Does your space inspire?” (1);

Large text on right (1);

Pictures on the far right column (1); Looked for what company about (would leave because didn’t find it) (1).

Continued




40 Initially thinks purpose: To create a better work environment (6);

To help managers make decisions (2);

Selling something, not clear what (2);

No idea (2);

Consultant clearinghouse (1); Commercial for healthy office environments (1); Reference information on various topics (1); Possibly selling planning or evaluation services (1); Shows need to be proactive about workspace (1).

41 Site intended for: Management (12) ;

Both management and workers (1); Separate parts different target audiences (1);

Management, HR, people in transportation, logistics, shipping, etc. (1); Commercial developers, interior designers, furniture suppliers, maybe employees (1); People who would buy or specify products (1).

42 Learn anything first 5 minutes? No (13); Yes (4). The “Yes”es tended to have to do with a better, but incomplete, understanding of what BetterBricks is about. “They don’t sell office equipment.” – P5.

43 Prompt to do anything? No (10); To try to figure out what it’s about (3); to continue exploring (2); not sure (1). None indicated that it motivated them to address workspace problems.

44 Too wordy. 1

45 Didn’t notice top bar of field studies. 8

46 Expected different audio clips for different views of the field studies.

1

Continued




47 Doesn’t understand, initially, how to use 360 degree view tool.

10 or more.

48 Learned employee retention & good work space relationship.

1

49 Would not use e-cards. Most.

50 Would prefer to fill-out “Rate your space” questionnaire online.

About half.

51 Afraid of junk mail if give e-mail. 1 stated explicitly; others had general concerns about giving information early in the process.

52 Impression of site improved over time. No (9); like it more (7). “At first, I thought it was an advice line. Now, I think it’s a consulting service” (1). Generally, the impression of the site improved over time. Many participants found valuable information, although almost all were frustrated by organizational problems and by not knowing what BetterBricks does.

53 Likes lots of links, places to explore. 1

54 At end, not sure what site about. All.

55 Thinks Advisor is information have to give to register to use the site.

1

56 Wants info on BB’s quickly. All or almost all.

57 Likes Tour. 1

58 Likes that site is not flashy. 1

59 Likes that there is lots of white space, large text.

1

60 Likes “Rate space” questionnaire. Most.

61 Likes “Building loyalty”, but should be called “Building team”.

1

62 F.S. of Herman Miller: about HVAC’s? 1

63 Can’t increase size of field study window. 2

Continued




64 Likes audio clips. 1

65 Likes field studies. Almost all.

66 Should tie tighter to contractors. 1

67 How BB’s tied to field studies? 2

68 What is “Discover” about? Should have a mission statement.

1

69 Likes “Act” area. About half.

70 Likes “Share with us”. No strong feelings.

71 Likes not full of ads. 1

72 Make serious TV commercials. 3

73 BB site aimed at top management. See 41.

74 Thinks site is primarily about old buildings. 1

75 Font too small. 5

76 Too much sales pitch. 1

77 Highlight key words. 1

78 Doesn’t want cubicle people to see this information: would get too many complaints about things he can’t fix.

1

79 Not sure who the site is aimed at. 1

80 Wants good numbers on productivity gains, financial gains.

Several.

81 Employee satisfaction is key in the Northwest.

1

82 Didn’t notice 360 degree view tool on field studies.

Most.

83 Want section on legal issues related to BB topics. Liability, etc.

1

Continued




84 Would fill out the Advisor form. A few. Most want more background information and a compelling reason first. Shorter form would get more responses.

85 Wants shorter Advisor form. All or almost all.

86 Saves money initially over asking questions to architecture firm.

1

87 Needs to highlight key information. 1

88 Colors are bad. 2

89 Seen TV ad too often, annoying. 1

90 Opening page “too garbagy” 1

91 Wants better explanations for big text on home page.

1

92 No idea what “does your space inspire” is about.

1

93 Good writing, good information. 1

94 Wants site map to have mouse-over description.

1

95 Wants “Better Bricks” as lead-off, with tour.

1

96 Intriguing, once figure out. 1

97 Used search facility. 1

98 Surprised that it doesn’t talk about office furniture.

1

99 Wants “home page” button. 1

100 “Advisor” label unclear. Call “Get more help” or such.

1

101 Likes field study fade-ins. 2

102 Problems with tabs & sub-tabs. 1

103 Wanted search engine, but didn’t see it. 1

Continued




104 Thinks Adv isor is registration materials. 1

105 Why two “Discover” links on the Tour page?

1

106 Likes the scroll bar. 2

107 Site map colors don’t match tab colors. 1

108 Wants technical information, not examples.

2

109 Can’t find the list of all field studies. Almost all, although not posed as explicit task.

110 Likes PowerPoint presentation. 1 (Only one to get through it.)

111 Deletes wrong window getting out of PowerPoint presentation.

1

112 Act ads are hard to read. 1

113 Thought “Act” would be a menu of services.

1

114 Didn’t get to any field studies on own. Several.

115 Didn’t notice BB name watching TV ads. Several.

116 Air quality is not a strong sell. 1

117 Likes “studies have shown”: indicates good information.

1

118 Make the small print at the top of the Advisor area easier to read.

1

119 Give clearer intro on Building Loyalty. 1

120 Building Loyalty: “What new facility?” 1

121 Building Loyalty: get rid of the middle picture, increase size of examples.

1

122 Healthy habitat: talk about it, but what is it?

1

123 Too much text in Tour. 1

Continued




124 “Great Spaces” should have an overview. 1

125 Thought that could fill out “rate your space” online.

Several. (Try to click on questionnaire boxes, puzzled that nothing happens.)

126 How do hall pictures help me understand the HVAC?

1

127 ----

128 Would send e-cards to friends. A few.

129 Make “Great Spaces” the starting point. 1

130 “Lighting Design”: there is a small about of text left over after paging down twice. Try to fit it into t wo pages, to save a mouse click.

1

131 Color match problems in Influence. 1

132 Want single click to move down a section for BetterBricks scroll bar.

1

133 Field studies are nice, but what action should I take?

1

134 Wants to be able to enter office layout for analysis.

1

135 No way to get out of 360-view facility without losing field study as well.

Inherent?





APPENDIX B: PARTICIPANT COMMENTS

Some of these comments are paraphrased.

Ø ”Fancy footwork that doesn’t go anywhere.” – Participant 1 (General Commercial) on the tabs at the top of the page.

Ø “Get rid of the Java fade-ins. They’re over-used and lack ‘wow’. For visual effects, I’ll go to the latest movie.” – P1.

Ø “They did a lot of work on this. Were the specs bad? Or was it done by someone not used to surfing the net?” – P1

Ø “This is a good example of what not to do in web site design.” – P1, in summary.

Ø “I don’t run to the computer and type in the web site because I thought the commercial was funny.” – Participant 3 (Influencer) on recommending more informative TV ads.

Ø “You should tailor this so contractors can bid for services through the web site.” – P3.

Ø “I don’t want the cubicle folks seeing this. There’ll be too many complaints about things I can’t fix.” – Participant 4 (Decision Maker) (in charge of facility).

Ø “This is a good place to start, before paying an architect to answer questions.” – P4.

Ø “I thought the Advisor section would be a list of consultants. Call it something like ‘Get More Help’.” – Participant 5 (Decision Maker).

Ø “There’s good information, but I sure wouldn’t know it from the home page.” – P5.

Ø “There’s a lot of useful information in here, if you can find it.” – P5



Ø “Make it very clear. ‘This is what we do. Here’s an evaluation tool. Here’s this … this … this … look at these.’ Don’t make me hunt for information.” – P5.

Ø “I want layers of detail. Provide definitions, then examples, then how to carry out changes.” – P5.

Ø “Great site, but ignore the first page.” – P5, on how to describe the site.

Ø “Who is the lady who is weird?” – P5, on why the TV ad does not inspire going to the site.

Ø “I have no idea what you do and the opening page sucks big time. Maybe you sell office furniture somewhere, I don’t know.” – P5, after approximately an hour on the site.

Ø “There’s too much to pick, I don’t know what to pick first. Where am I supposed to start?” – Participant 6 (Influencer).

Ø “Why aren’t they talking about Feng Shui?” – P6.

Ø “My first impression is that they are a furniture manufacturer.” – Participant 8 (Decision Maker).

Ø “I want to know quickly: who are you, and what do you do?” – P8.

Ø “The home page has to say what you do or I won’t go any further.” – P8.

Ø “I’ve lost interest.” – P8

Ø “Awkward, confusing, not to the point right away.” – P8, describing web site.

Ø “How far do I have to go to find out what this is about?” – Participant 9 (Architect).

Ø “I want to see an opening statement, a big picture about what the site is about, then move to details and options.” – P9.

Ø “As an architect, I like visual images, but I want them tied more directly to environmental issues. If you are talking about lighting, describe unique lighting approaches. Also, provide Internet links to other resources, studies and reports backing up environmental concepts.” – P9.



Ø “Tell ‘em what you’re going to tell ‘em, tell ‘em, then tell ‘em what you told ‘em.” – P9, on web site design.

Ø “Who are you, and how do I make contact?” – Participant 10 (Developer).

Ø “Pretty good! Very neat!” – P10, on 360 degree views.

Ø “Get the site endorsed by landlords, property managers, brokers and real estate agents. Show how the material in the web site produces higher value.” – P10.

Ø “Why do they want all this information? Why should I be contacting them? What will I get back? Just a bunch of e-mail?” – Participant 11 (Developer), on Advisor.

Ø “Why do they ask me my reason for contacting them, when they tell me above why to contact them? Is there an ulterior motive?” – P11, on Advisor.

Ø “Good information, but you have to work to get it.” – P11.

Ø “Up-front, say who you are and what you offer. Then guide me through it. If it’s about HVAC, etc., go there. I still don’t know what you offer.” – P11.

Ø “Set the site up as ‘What we’re about’, then options.” – Participant 12 (Influencer).

Ø “I’d like to have a spot like: having light problems? What’s the cause? How does it hurt me? What should I do about it? Lead me in a direction.” – P12

Ø “For instance, in the ‘rate your space’ section, where there’s a problem, make suggestions about what to do about it. A request for changes based on vague information just annoys the boss. I need specifics.” – P12.

Ø “The further in I get, the more confusing it gets. Are you selling furniture?” – Participant 13 (Architect).

Ø “It beats around the bush stunningly. It’s not telling me quickly enough what it is proposing. Maybe not at all, I don’t know.” – P13

Ø “There’s too much text. I can’t get in and out with information quickly. I’m slowly spinning in: by the time I get there, I’ll be too old and dilapidated to enjoy it.” – P13



Ø “This site is about how to make higher quality work time, without the cheesy girl.” – Participant 14 (Influencer).

Ø “It kinda moves really fast. Like, oh-oh, what do I do?” – P14, on tabs at the top.

Ø “Definitely neat. I like this.” – P14, on 360-degree views.

Ø “At first, I thought it was about Internet services. Now it seems to be about office layout.” – Participant 15 (Decision Maker).

Ø “I still don’t know what inspired this company to exist.” – P15.

Ø “Would I spend the time to research this site? I try not to get sucked into 20 second look and go… but if the information is not easily there, it’s a turn-off.” – P15.

Ø “Use bookmarks, not posters. Posters don’t get seen.” – P15, on Influence.

Ø “The first thing to do is tell me is, ‘who is BetterBricks?’ About number ten is, ‘now can we ask you for information, after we’ve done something for you?’ In between, give me a guided tour.” – P15

Ø “Though our culture is incredibly independent, we still want to be told where to go. Start on page 1, take us through the steps.” – P15

Ø “This web site was generated by very creative people. It’s flashy and shows a lot of work. But I’m short on time. I want it to get to the point, narrow in on what my needs are.” – Participant 16 (General Commercial).

Ø “My hand goes automatically to the right to scroll. I have to force myself to use their scroll.” – P16

Ø “This is intimidating. I wouldn’t fill it out. I don’t see a privacy statement. Some of this I couldn’t even answer without an NDA.” – P16, on Advisor section.

Ø “Just ask for an e-mail and phone number and say, ‘we’ll get back to you.” – P16, on Advisor section.

Ø “Will this be slow at 28k?” – P16. [Even though he has a fast link at work, about 50% of the time he looks at sites over his home modem. He pointed



out that many companies give old computers to their employees, meaning that they will tend to have lower-end systems at home.]

Ø “Why is less personal information required in ‘Share With Us’ than ‘Advisor’? Do you not need to know as much when you’re asking me for a favor?” – P16, who recommends combining the two areas.

Ø “If I can find specific topics relevant to my work life, it would motivate me to contact the firm.” – P16.

Ø “For the field studies, provide specific costs, timeline, how many people involved, etc. Cost information up front does not discourage real customers. Provide more depth on the process, as well as the results.” – P16.

Ø “I’d like to have the characters from the ad on the picture on the home page.” – Participant 17 (General Commercial).

Ø “This web site is better than the Late Night News.” – P17, who was enthusiastic about bringing it up as an information source in meetings with management.

Ø “Cool! This is like being there.” – P17, on 360 degree view.





APPENDIX C: USABILITY TEST PROCEDURES

FINAL

USABILITY TEST PROCEDURES

FOR

BETTERBRICKS.COM by

Jerrold Prothero, Ph.D.

Hypercerulean, Inc.

August 10th, 2000

Seattle





USABILITY TEST PROCEDURES

Usability tests help to find where users are having problems, by analyzing the performance of individual users on realistic tasks. Participants talk aloud as they go through the test, to provide a record of what they are thinking. The three phases below aim to gather information about first impressions; ability to perform tasks; and opinions about the web site after having become familiar with it.

Since usability testing is intended to find as many interface problems as possible as quickly as possible, the below procedures may change depending on what is found in early trials.

Background

The participants will fall into four categories of potential users of the betterbricks.com web site: general commercial employees; decision makers of commercial firms; influential employees of commercial firms; and supply-side providers of new commercial buildings – designers and developers. (See the user profiles for further information.)

The usability test will be conducted with sixteen participants, eight in Seattle, and eight in Boise (two from each of the above categories at each site).

Goal: Examine the betterbricks.com site for potential usability problems, focusing on comprehension and ability to use the site to raise awareness and to help make or influence building design decisions. Also, after getting usability data, ask marketing-related questions. This provides marketing information from participants who have become very familiar with the betterbricks site.

Participant Introduction

Introduce usability testing in general, get informed consent, warm up.

Set the scenario as the participant having come across the betterbricks.com site after seeing a both betterbricks.com TV ad. Play video. Initially, do not further explain the purpose of the site to the participant, so we can determine how well they grasp it on their own.



PHASE ONE: FIRST IMPRESSION

Phase One examines critical first impressions. Web visitors have notoriously short attention spans. If the first impression is not favorable, there usually will not be a second impression.

Participant is given 5 minutes to explore the site on their own, while talking aloud about what they are thinking and doing. Then they are asked (with additional non-leading follow-up questions as appropriate):

Q1.1: Was this site what you expected from the TV ads you saw?

Q1.2: What were your first thoughts about it? What are your thoughts now?

Q1.3: What first drew your attention?

Q1.4: Where do you think you would spend the most time on the site?

Q1.5: How would you describe the purpose of the site?

Q1.6: Who do you think this site is intended for?

Q1.7: At this point, how likely would you be to want to continue exploring the site? Would you come back for a second visit? Why or why not?

Q1.8: Did you find out anything new from your first five minutes on the site?

Q1.9: Did you feel the site prompted or encouraged you to do anything?

PHASE TWO: TASKS

Phase 2 uses various tasks to help the respondent further evaluate the site. Task 2 is an extended exploration of the site. Task 3 asks how they would use the site to guide their own or other people’s decisions. Task 4 focuses on specific features participants may not have visited in the above. (Since we have a bit over an hour of participant time to work with: on a participant-by-participant basis, Task 4 will be shorted or dropped as needed.)

Task Two: Extended Exploration

T2.1: “Suppose you wanted to get a more detailed understanding of this site. How would you go about doing this?”



[Observe how they explore the site.]

Q2.2: “ What are your thoughts about the site at this point?”

Q2.3: Did you find out anything new during this exploration? If so, what?

Q2.4: Did your impressions of the site change at all?

Task Three: Gathering Decision-Related Information

{Show them the print ads appropriate to their participant category.}

T3.1: [Specialized depending on participant category.]

Worker Categories: Suppose your company was planning a new building or renovating your work space and you wanted to convince (management, yourself, coworkers) of the importance of the physical work environment. How might you use this site to convince them? (Do you feel you could use anything on this site to help convince them?)

Architect/Developer: Suppose you are working with a client (or, for architects, a developer) on a new building and you wanted to convince them about the importance of the physical work environment. How might you use this site to convince them?

[Observe how they use the site.]

Task Four: Test Specific Features or Areas

T4.1: Have influential, general employee categories only fill out the survey. Follow-up questions as appropriate.

T4.2: Show influential, decision-maker, architect/developer only the Advisor area. Follow-up questions as appropriate.

T4.3: Take all through a case-study. (Possible exception: influentials since they might have to much to do already) Case studies to be determined by Heather at Cole Weber. Follow-up questions as appropriate.

PHASE THREE: POST-TEST QUESTION

These Questions are asked of the participant after the above tasks are completed. This is an opportunity to find out what someone with about a hour of intense experience with the site thinks about it.



Q5.1: Have your impressions of the site changed since you began your visit? How? Q5.2: Did you already know about the information at this site, or did you find out anything new? If new, what was it?

Q5.3: “Would you do anything as a result of visiting this site? If so, what?”

Q5.4: “How would you describe this site to a (boss, colleague, client)?”

Q5.5: “Would you recommend this site to a (boss, colleague, client)? Why or why not?”

Q5.6: “What do you like about this site?”

Q5.7: “What don't you like about this site?”

Q5.8: “What would you change or add?”

Q5.9: [Follow-up on Q5.8. if not mentioned.] “Would you like to see more detailed information? More case studies? How to carry out changes? What the financial benefits would be?”

Q5.10: “To what extent do you think the information on this site is accurate, complete and unbiased? Why do you say that?”

Q5.11: “Why do you think betterbricks.com is providing this web site? Do you think betterbricks is a for-profit or a non-profit organization? Would the difference matter to you? Would you like the site to contain more information about betterbricks.com?”

Q5.12: Thinking back on both the TV and printed ads, how well do you think they introduced the web site?

Q5.13: “Is there anything else you would like to pass on to the betterbricks.com team/sponsors?”

Thank you for your input.

If you have questions or suggestions contact:

Jerrold Prothero, Ph.D. Hypercerulean, Inc.

(206) 669-7094 - [email protected]



APPENDIX D: BACKGROUND ON HUMAN COMPUTER INTERFACES

I: Why HCI Matters

The area of Human Computer Interfaces (HCI) is an emerging new field that deals with the question of how humans and computers can interact more effectively. One reason for its importance is that, regardless of where most of the sophistication in a system lies, for most users the interface is the system. Consequently, a poor interface can leave an otherwise excellent system with a poor reputation and few users.

However, interface problems do not merely affect users' perceptions: they also affect their performance. In the absence of systematic attention to the interface, user performance is usually far from optimal. According to research cited in the recent National Research Council report "More than Screen Deep: Toward Every-Citizen Interfaces to the Nation's Information Infrastructure", interfaces can generally be greatly improved by a test-and-redesign methodology. "It is by now well-established that iterative test-and-redesign methods almost always result in substantial gains in usefulness and usability. Landauer summarizes a large number of published results of comparisons of task performance efficiency before and after a redesign based on some kind of empirical evaluation of the efficiency of a computer-based system. The modal gain is around 50%, and new methods and new evidence of their success appear regularly."

HCI at its far cutting edge deals with topics such as virtual reality and wearable computers. More immediately, it deals with making systems easier to use. In recent decades, increasing computational power has allowed computers to come closer to using intuitive human modes of interaction, rather than forcing humans to conform to the idiosyncrasies of computers. For instance, 1-dimensional (command-line) interfaces have been extended to include 2-dimensional (windowing) and 3-dimensional (virtual) interfaces, which make better use of human visual capabilities.

The increased computational power now available creates the potential for interfaces that greatly improve the effectiveness with which humans can access and use information. However, this increased computational power, along with increased information flow, means that interfaces have become highly complex. This is further exacerbated by the fact that most software developers are not typical of the intended end-users, and consequently may have difficulty predicting where end-users will experience difficulties. Viewed from the perspective of those familiar with a system, "people are incredibly creative in generating errors and misconceptions, and incredibly fast.” (Carroll & Aaronson)



II: General Interface Design Principles

This section summarizes some of the principles that have been formulated for effective interface design. While these principles do not give detailed guidance in specific cases, they are worth keeping in the back of one's mind.

Norman mentions four principles of good design:

§ The state and action alternatives should be visible.

§ There should be a good conceptual model with a consistent system image.

§ The interface should include good mappings that reveal the relationships between stages.

§ The user should receive continuous feedback.

Norman's principles indicate the idea of transparency: the user should have the feeling that he/she is looking straight through the interface to the problem of interest, almost as if the interface wasn't there at all.

Shneiderman gives the following interface design principles:

§ Strive for consistency.

§ Enable frequent users to use short cuts.

§ Offer informative feedback.

§ Design dialogs to yield closure.

§ Offer simple error handling.

§ Permit easy reversal of actions.

§ Support internal locus of control.

§ Reduce short-term memory load.

In summarizing several collections of design principles, Dumas and Redish point to the following commonalities:

Give the user control.

Strive for consistency.



Smooth the human-computer interactions with feedback.

Support the user's limited memory.

III: Menu Organization

A number of studies indicate the importance of categorical, rather than alphabetic or random, menu organization. Shneiderman concludes that "These results and the SSOA model suggest that the key to menu-structure design is first to consider the semantic organization that results from the task. The task terms and structure come first; number of items on the display becomes a secondary issue."

A second finding is that broader, shallower menu trees are to be preferred over deeper, narrower trees. "In an interesting variation, Wallace et al. confirmed that broader, shallower trees (4 x 3 versus 2 x 6) produced superior performance, and showed that, when users were stressed, they made 96 percent more errors and took 16 percent longer. The stressor was simply an instruction to work quickly...” (Shneiderman).

IV: Short-Term Memory

A classic paper in the psychology of memory is Miller. This paper indicated that the maximum number of distinct units of information that humans can keep track of simultaneously is between 5 and 9. This suggests that interface designers should be careful to minimize the number of things which users have to be able to track simultaneously.

An interesting phenomenon is that with familiarity, common patterns of information tend to be processed as single large chunks, each of which can fit into a single slot of the 5-9 slots available in short-term memory. Consequently, experts who are familiar with a system and who have formed many chunks can keep much more information in short-term memory than can novices. This is a major contributor to the superior performance of experts over novices. This difference in memory capability should be remembered by interface designers, who tend to be experts developing for novices.

Another important consideration is that items can only stay in short-term memory for about 15-30 seconds without being used before they are forgotten. This has important consequences, for instance, to the design of help features. If it takes users more than about 30 seconds to find help relevant to the problem they were working on, they will tend to forget why they needed the help, and have difficulty applying the help information.



V: Response-Time Delays

The length of time it takes a system to respond to user input can qualitatively change user performance and satisfaction. As quoted from Shneiderman, "Briefly, if delays are long, users will seek alternative strategies that reduce the number of interactions, whenever possible. They will fill in the long delays by performing other tasks, daydreaming, or planning ahead in their work. These long delays may or may not increase error rates when they are in the range of 3 to 15 seconds, but they probably will increase error rates if they are above 15 seconds if people must remain at the keyboard waiting for a response. Even if diversions are available, dissatisfaction grows with longer response times."

VI: The Use of Color

While color can be an aid in grouping related information, it can be quite distracting if overused. A useful rule is to limit the number of colors in a single page to four, with a maximum of seven colors used throughout. However, more experienced users may benefit from the use of more colors to indicate finer relationships. Color-coding should be used consistently and in the support of critical tasks. The relationships indicated by the color code should be apparent to the user with a minimum of effort.

Hypercerulean, Inc.

Seattle

contact for questions: (206) 669-7094

EBPI – MPER #1 DETHMAN & TANGORA h APPENDIX E

Appendix E: Final EBPI Codes Support Process Evaluation Interview Guide


DETHMAN & TANGORA h EBPI – MPER #1 APPENDIX E

EBPI – MPER #1 DETHMAN & TANGORA h E - 1

FINAL EBPI CODES SUPPORT

PROCESS EVALUATION INTERVIEW GUIDE

NORTHWEST ENERGY EFFICIENCY ALLIANCE

Name

Project

Date ___________________________/Phone number:

Introduction (NOTE: Tell/remind them David Cohan at the Alliance will also be contacting them for developing a code support strategy for the future.)

Initial recruitment (note: guideline language – tailor as needed): Hello, this is ______________ . I’m working with Dethman & Tangora, a research and evaluation firm in Seattle, WA. The Northwest Energy Efficiency Alliance has hired us to help them assess its funding, through the Efficient Building Practices Initiative, for Energy Code Support in Washington, Oregon, and Idaho. I believe you have a contract with the Alliance for Energy Code Support, is that correct? Are you the right person to talk with?

We’d like to gather your opinions on how well this support has worked, the need for continuing support, and how such support can be improved if it’s offered in the future. You should know that all your specific input is confidential, unless you give us permission otherwise, and will be used only for evaluation purposes. Can I set up a time to talk with you? We should plan on about 45 minutes to an hour interview.

Actual interview: (Introduce self) As you recall, I’d like to talk with you today to gather your opinions about the Energy Code Support you’ve received from the Northwest Energy Efficiency Alliance.


1. First, please tell me your job title, the agency you work for, and a bit about your background.


DETHMAN & TANGORA h EBPI – MPER #1 E - 2

2. Could you briefly tell me your understanding of why your organization asked for Energy Code Support from the Alliance?

3. And could you tell me, in general, the overall goals you wanted to achieve through this funding? (Probe: How effective have you been in meeting these goals? Why?)

4. I’d like to know if you were trying to meet any of the following goals with the funding you received from the Alliance.#

a. Helping to increase public support and awareness of the benefits of energy codes?

Yes No Don’t know

If yes: How effective do you feel you’ve been you been in meeting this goal?

Very Effective

Somewhat Effective

Not Too Effective

Why do you give that rating? (Probe for specific things they did that would demonstrate effectiveness) b. Facilitating a transition to a permanent, sustainable code support structure

Yes No Don’t know

If yes: How effective have you been in meeting this goal?

Very Effective

Somewhat Effective

Not Too Effective

Why do you give that rating? (Probe for specific things they did that would demonstrate effectiveness)



c. Helping to mainstream energy codes within the building codes.

Yes No Don’t know


Very Effective

Somewhat Effective

Not Too Effective

Why do you give that rating? (Probe for specific things they did that would demonstrate effectiveness) d. Encouraging the support and participation of partner organizations in this effort.

Yes No Don’t know


Very Effective

Somewhat Effective

Not Too Effective

Why do you give that rating? (Probe for specific things they did that would demonstrate effectiveness) e. Making strategic investments to maximize co-funding and participation by other institutions as well a private sector organizations.

Yes No Don’t know


Very Effective

Somewhat Effective

Not Too Effective

Why do you give that rating? (Probe for specific things they did that would demonstrate effectiveness)

Funding and Work Activities

5. From your point of view, how essential has the Alliance funding been in terms of maintaining energy codes in your state? Improving energy codes in your state? (Probe: Why do you say that?)

Maintaining?



Improving?

6. What other funding, if any, has been used to support energy codes maintenance during the time that you’ve had Alliance funding? Energy code improvement? (Probe: How much has that funding been?)

Maintenance?

Improvement?



7. Now I’m going to read you a list of code-related work activities. Please tell me if the Alliance is sponsoring that type of work in your office. (Probe for each: What percent of that activity is funded by the Alliance, and what other funding supports that activity?)

Sponsoring? % funded by Alliance

Other funding?

Revisions and amendments to upgrade or improve energy code language

Yes No DK

If yes, what have you done (or intend to do)? Is there more to be done? Responding to energy code inquiries from building departments and applicants

Yes No DK % funded by Alliance

Other funding?

If yes, what have you done (or intend to do)? Is there more to be done? Tracking and identifying recurring code implementation and interpretation issues


Other funding?

If yes, what have you done (or intend to do)? Is there more to be done?



Improving energy code application and processing materials, such as forms


Other funding?

If yes, what have you done (or intend to do)? Is there more to be done? Improving field inspection tools Yes No DK % funded by

Alliance Other funding?

If yes, what have you done (or intend to do)? Is there more to be done? Conducting training and education on energy codes


Other funding?

If yes, what have you done (or intend to do)? Is there more to be done?



8. Besides the types of work activities we’ve just talked about, are there other work activities the Alliance is funding that we haven’t talked about? Why was funding that work important?

Codes Viewpoints

9. I’d like to read you a list of statements that might guide a long term strategy for maintaining and improving energy codes in the Northwest. I’d like you to tell me whether you think each one is a high priority, a medium priority, or a low priority for your state at this point in time.

High Priority

Medium Priority

Low Priority

Energy codes should be simple for designers to apply to their buildings

Energy codes should be simple for building departments to enforce

There should be a good balance between design flexibility and enforceability

The long term need is to keep stringency levels abreast of building technology and design practices through periodic updates

Continuing education, both for designers and for building departments, is important to maintain and improve effectiveness of energy codes

Building departments need sufficient resources to make energy codes work

The private sector has an important interest and role in assuring effective energy codes

Energy codes should be mainstreamed with the rest of building codes

Energy codes should become more consistent between states and regions, without losing the strengths of existing state codes

Energy code financial support needs to receive permanent funding



Relationship With the Alliance

10. Now I’d like you to rate the Alliance on the following aspects of the funded project’s operation? (Probe if any problems indicated)

The clarity of the initial request for proposals Excellent Good Fair Poor

Contracting procedures Excellent Good Fair Poor

Monitoring of the project Excellent Good Fair Poor Clarity about deliverables and time schedules Excellent Good Fair Poor Communications with the Alliance & project contact

Excellent Good Fair Poor

Connection to other aspects of energy code support in the region

Excellent Good Fair Poor

Payment procedures Excellent Good Fair Poor

11. If the funding were to continue, what improvements would you want to make in that relationship? (Probe)

Final/Wrap-Up Questions

12. In your opinion, what have been the greatest benefits of the Alliance funding? What have you been able to do that you wouldn’t have been able to do without it?

13. What could be improved about this funding arrangement? How could each of these changes best be made?

____________________________________________________________________________



14. Overall, how optimistic are you about maintaining and enforcing energy codes in your state over the next few years?

Very Somewhat Not Too Not At All Optimistic Optimistic Optimistic Optimistic

15. Why do you give that rating?

16. Over the next few years, do you feel energy codes in your state face a . . .

1 Period of relative stability

2 Period of moderate change

3 Period of substantial change

17. What are the two or three most important issues facing the maintenance and improvement of energy codes in your state in the next few years? Please be as specific as possible. (Probe: Likelihood that energy code support will be self-sustaining.)

18. Would you like to be able to receive further funding from the Alliance when this contract ends? Why or why not?

19. (If funding desired) If you didn’t get further funding, what is likely to happen to energy code maintenance and enforcement in your state?



20. Finally, what’s the most important piece of advice you can give the Alliance as it seeks to support the maintenance and revisions of energy codes in the region?

Thank You for all your help. (Remind about David Cohan)

EBPI – MPER #1 DETHMAN & TANGORA h APPENDIX F

Appendix F: State Special Projects Process Evaluation Interview Guide


DETHMAN & TANGORA h EBPI – MPER #1 APPENDIX F


EBPI – MPER #1 DETHMAN & TANGORA h F - 1

STATE SPECIAL PROJECTS

PROCESS EVALUATION INTERVIEW GUIDE

NORTHWEST ENERGY EFFICIENCY ALLIANCE

Name

Project

Date ___________________________/Phone number:

Introduction

Initial recruitment (note: guideline language – tailor as needed): Hello, this is ______________. I’m working with Dethman & Tangora, a research and evaluation firm in Seattle, WA. The Northwest Energy Efficiency Alliance has hired us to help them assess the progress of their State Special Projects, funded under EBPI. I believe you have (or had) a State Special Projects contract called the ________________________________with the Alliance, is that correct? Are you the right person to talk with? We’d like to gather your opinions on how well this project has gone and collect any insights you might have about lessons learned and improvements for the future. You should know that all your specific input is confidential, unless you give us permission otherwise, and will be used only for evaluation purposes. Can I set up a time to talk with you? We should plan on about 45 minutes to an hour interview. Actual interview: (Introduce self) As you recall, I’d like to talk with you today to gather your opinions about the ______________________ funded by the Northwest Energy Efficiency Alliance. (Explain that I’ve read some background information about the project but may not have full information about it.)

Background and Goals of the Projects

1. First, please tell me your job title, your organization, and a bit about your background.


DETHMAN & TANGORA h EBPI – MPER #1 F - 2

2. Could you briefly explain your project to me and its current status?

3. Please tell me your understanding of why your organization asked the Alliance to fund this project? (Probe: Why was this project important?)

4. And could you tell me the overall goals you wanted to achieve as a result of this project? Probe for each: How effective have you been in meeting these goals? Why do you say that? What examples might best illustrate how effective you’ve been?



5. The Efficient Building Practices Initiative, through the State Special Projects and other activities, had three major purposes: One purpose was to: (Note: use appropriate tenses depending on the project status)

Generate innovative ideas to support energy codes. Would you say your project helped meet that purpose of EBPI? Probes: If so, how did it support energy codes? What, if anything, do you feel was innovative about your project’s approach to supporting energy codes?

A second purpose was to promote voluntary building energy efficiency. Would you say your project helped meet that purpose? Probes: If so, how did your project promote voluntary building energy efficiency? What was most effective in that effort?

A third EBPI purpose was to promote partnerships in the building industry? Would you say your project helped meet that purpose? Probes: If so, what did you do to promote partnerships? What was most effective for that effort?

Funding and Leveraging

6. From your point of view, how essential has the Alliance funding been in terms of undertaking this project? What have you been able to do that you wouldn’t have been able to do without it? (Probe: Why do you say that?)



7. What other funding, if any, has been used to fund this project during the time that you’ve had Alliance funding? (Probe: How much has that funding been?)

Relationship with the Alliance

8. Now I’d like you to rate the Alliance on the following aspects of their involvement with this project? (Probe if any problems indicated)

The clarity of the initial request for proposals Excellent Good Fair Poor DK/NA

Contracting procedures Excellent Good Fair Poor DK/NA Monitoring/reporting requirements Excellent Good Fair Poor DK/NA Clarity about deliverables and time schedules Excellent Good Fair Poor DK/NA Communications with the Alliance Excellent Good Fair Poor DK/NA

Help from the Alliance when you needed it Excellent Good Fair Poor DK/NA Payment procedures Excellent Good Fair Poor DK/NA

Extra comments:

9. What improvements, if any, would you want to make in that relationship? (Probe)

Final/Wrap-Up Questions

10. In your opinion, what have been (are) the greatest benefits or indicators of success for your project?



11. What would you have liked to have improved about your project?

12. Are there future projects of this type that you would like the Alliance to fund? What are they and why do you feel they’re qualify?

13. Finally, what’s the most important piece of advice you can give the Alliance as it seeks to support energy codes in the region, promote voluntary building energy efficiency, and promote partnerships in the building industry?

Thank You for all your help.



EBPI – MPER #1 DETHMAN & TANGORA h APPENDIX G

Appendix G: Final Baseline Evaluation Report Efficient Building Practices Initiative –

Executive Summary

Appendix G: Final Baseline Evaluation Report EBPI – Executive Summary

DETHMAN & TANGORA h EBPI – MPER #1 APPENDIX G

EBPI – MPER #1 DETHMAN & TANGORA h G - 1

FINAL BASELINE EVALUATION REPORT EFFICIENT BUILDING PRACTICES INITIATIVE – EXECUTIVE SUMMARY

BACKGROUND

The Northwest Energy Efficiency Alliance (Alliance) is a non-profit group of electric utilities, state governments, public interest groups and industry representatives committed to bringing affordable, energy-efficient products and services to the marketplace. The Alliance’s Efficient Building Practices Initiative (EBPI) couples a Regional Public Information Program (RPIP), with several building code support programs to:

Ø Increase the demand for highly energy efficient residential and commercial buildings.


This overall goal of evaluating EBPI is to assess how well and to what extent this dual approach works to change markets for energy efficient buildings.

This Executive Summary focuses on the results of RPIP evaluation activities to date. It provides key findings and conclusions from baseline market surveys of primary and secondary target audiences for RPIP’s initial public information campaign, BetterBricks.com. Currently this campaign includes advertising, public relations, marketing, a website, a help desk, and design guidelines. Chapters 3 and 4 of this report provide detailed findings from these baseline surveys.

The rationale behind the BetterBricks.com campaign is to forge a stronger awareness, among target audiences, of the link between job productivity and the benefits of energy-efficient workplace elements which would, in turn, encourage demand for higher efficiency commercial buildings.

To set the stage for the overall EBPI evaluation, this report also presents:

1. An overview and history of EBPI (Chapter 1)

2. The EBPI evaluation approach (Chapter 2)

3. Key indicators for RPIP’s BetterBricks.com campaign (Chapter 5)


DETHMAN & TANGORA h EBPI – MPER #1 G - 2

4. Conclusions about the BetterBricks.com efforts (Chapter 6)

KEY FINDINGS

Baseline Assessment Of Commercial Workers

To establish a market baseline for commercial workers, we surveyed three primary target audiences of the BetterBricks.com public information campaign: general employees, influential staff who influence workspace decisions, and decision-makers who are in charge of workspace decisions. The surveys investigated key company characteristics, opinions about the effects of the physical workspace on productivity, actions taken to improve productivity, sources of information about workspace and productivity, and Internet use and TV viewing patterns, including whether they had seen any advertising or news stories relating the workspace to productivity.

The sample was drawn from a list of companies with at least 20 employees in Washington, Idaho, Oregon, and Montana. The evaluation team worked with Alliance staff to design the surveys, and Gilmore Research, Inc. conducted the interviews between April 13 and May 1, 2000, the launch date of the BetterBricks.com campaign. Each group had a sample size of about 500. (Please refer to Appendix A for details about the sampling design.)

Key characteristics of the respondents and companies include:

Ø Respondents represent workers in offices, schools, wholesale and retail stores, health care, and financial institutions.

Ø Most workers are in older buildings, with less than 20% in buildings less than five years old.

Ø Just over 60% of the companies occupy space that they own.

Ø Just over a third of decision-makers (35%) reported their companies planned to make or were in the process of making changes to their workspaces, including renovating current space, constructing their own building, or moving to new leased space

Ø Companies housed in older buildings and larger companies were more likely to be making space changes than those in newer buildings and smaller companies.



Key attitudinal and behavioral findings related to productivity and workspace include:

Ø Influential staff and decision-makers tended to rate their organizations more positively than general employees in terms of their companies “being concerned about worker productivity.”

Ø About 60% of respondents overall believe the physical workspace strongly affects productivity.

Ø When asked to name what physical factors had a large effect on employee productivity, all types of workers most often said the “layout of the workspace” (about 45% of each group), with “equipment” a distant second (about 22%). In addition, most steps already taken to improve productivity through physical changes involved layout and equipment changes.

Ø Although asked to confine themselves to “physical factors” related to productivity, respondents also noted the importance of non-physical factors such as management, co-workers, and compensation.

Ø Between 10% and 20% of respondents spontaneously mentioned workspace design features relevant to energy efficiency and of interest to EBPI – natural light, good lighting, good temperature – as strongly influencing productivity.

Ø When spontaneous mentions are added to prompted ratings of features relevant to energy efficiency, most workers said those features had a large effect on productivity (90% for lighting; 75% for temperature; 55% for natural light).

Ø Among the decision-makers who were planning or making changes in their workspaces, “increased productivity” was rated as the top priority among six potential reasons for changing space (79% important or very important ratings). Notably, a much smaller portion of architects named increased productivity as something decision-makers frequently mention as a reason to make workspace changes (46%).

Key findings related to information sources, media use, and recall of advertising related to a productivity website include:

Ø When asked to name the information sources they would use to learn more about the physical workspace and employee productivity, decision-



makers said they would use external sources. They most often mentioned reference materials—including the Internet—followed by equal mention of professional organizations, seminars, and design and construction firms.

Ø In contrast, employees most frequently mentioned they would seek information from within the company, turning to their managers and appropriate coworkers, followed by less frequent mention of seeking information from reference materials (including the Internet).

Ø Influential staff fell mid-way between the other two groups reporting, with equal frequency that they would turn to external reference materials and to internal staff.

Ø About 20% in each of the three groups would turn to the Internet for more information on the relationship between the workspace and employee productivity.

Ø About 70% of general employees, influential staff, and decision-makers report using the Internet on the job, with more than half of these users reporting that they access the Internet for four hours or less per week

Ø Over 95% of respondents typically watch some TV during the week. About 90% of respondents watch the news during the week, about 80% watch prime time TV, and fewer than 60% watch sports.

Ø Only about 5% of respondents reported they had seen advertising or news stories about a website providing information on improving productivity in commercial buildings. Most of this 5% could not describe anything specific about what they had seen, suggesting that few respondents (if anyone) had seen the first day of BetterBricks.com advertising.

Baseline Assessment of Commercial Architects and Developers

Architects and developers of commercial space are key players in generating, understanding and meeting the demand for buildings with high-energy efficient features. These groups were secondary target audiences of the BetterBricks.com efforts. We interviewed over 100 commercial workspace architects and developers in the Pacific Northwest during May 2000, exploring their experiences with and opinions about buildings that enhance productivity. These interviews occurred during the first two weeks after the BetterBricks.com launch (making it possible that some respondents might have noticed the campaign).



We drew the sample of architects from the population of members of Pacific Northwest chapters of the American Institute of Architects (AIA), as of 1998. We drew the sample of developers from a purchased list of businesses whose primary SIC code corresponded with development.

Key findings from these interviews include:

Ø Architects and developers play somewhat different roles with respect to commercial workspace design. Both architects and developers, however, are equally concerned with the building envelope and the general mechanical systems of interest to EBPI (e.g., natural light and temperature control). They are also equally concerned about lowering operating costs.

Ø Architects are likely to be more concerned than developers about the “look” and aesthetic appeal of the buildings and workspace, meeting energy efficiency and environmental goals, and dealing with interior lighting design and layout. They are also much more likely than developers to address the “human” factors of occupying a building (e.g., physical comfort, productivity, worker satisfaction).

Ø Seventy percent of architects spontaneously mentioned good lighting when asked to name the features of the physical workspace that contribute most to employee satisfaction and productivity. About 60% mentioned natural light and about 50% mentioned a comfortable temperature. Developers mentioned these features less often, with about 40% of developers naming natural light and comfortable temperature and about 30% naming good lighting.

Ø Architects, more than developers, were able to name specific natural lighting and artificial lighting strategies that they used to enhance employee satisfaction and productivity, but both groups were equally capable of naming heating and cooling equipment that would improve productivity. Both groups often said that good temperature control was one of the hardest objectives to achieve in commercial buildings.

Ø Architects would be most likely to turn to reference materials (61%), including the Internet, to learn more about the relationship between workspace design and worker satisfaction and productivity. The top three sources of information most frequently cited by architects were professional journals and magazines (41% a subset of reference materials),



professional organizations, seminars, and colleagues (35%), and the Internet (28%).

Ø Developers would be most likely to turn to architects and other design and construction professionals to learn more about how to design workspaces to enhance worker satisfaction and productivity. However, given their focus on the building envelope and technical systems, and not on the interior workspace, many developers suggested that it was unlikely that they would pursue such information.

Ø Almost 1/3 of architects and developers said they had seen advertising or news stories about improving worker satisfaction and productivity through workspace design. None, however, were able to specifically identify the just recently launched BetterBricks.com advertisements. This suggests that the BetterBricks.com campaign themes are familiar to them through other information sources.

CONCLUSIONS AND ISSUES

This overall goal of this baseline is to establish key performance indicators for the RPIP’s BetterBricks.com campaign, so that subsequent evaluations of this RPIP effort can be compared against them. These indicators are described in Chapter 5 of this report. Although the baseline research was not meant to generate recommendations, four conclusions and issues of concern to the RPIP’s BetterBricks.com campaign are summarized below. They are further discussed in Chapter 6.

1. The BetterBricks.com message is competing with other messages.

A challenge for RPIP’s BetterBricks.com campaign is likely to be the variety of sources, and the ideas and issues contained in those sources, that target audiences draw from when they talk about workplace productivity. Testing whether the messages of BetterBricks.com are heard amidst the already existing concern and interest in workplace productivity is one purpose of this evaluation.

2. BetterBricks.com messages may not currently attract developers.

Our research shows that developers often are not concerned with the design of interior workspaces; rather, they are involved with building shell and infrastructure



decisions. The current BetterBricks.com advertising campaign emphasizes interior design, which may not “hook” developers.

However, providing commercial building occupants with reliably comfortable temperatures is very important to many developers and is often difficult. BetterBricks.com messages will be looked upon more favorably by this group if it can help solve that problem for them. Developers also want reassurances that the innovations promoted by the BetterBricks.com campaign are bankable.

3. Primary target audiences prefer news and prime time TV to sports.

When considering ad placement, news and prime time may be better venues than general sports to reach the primary target audiences of general employees, influential staff, and decision-makers.

4. Preferences for information sources may vary by target audience.

Employee, influential staff, and decision-maker preferences for information sources about productivity and the workspace reflect their differing positions within organizational hierarchies. Employees reported that they would likely turn to internal staff to learn more about having a more productive workspace. In contrast, decision-makers said they would likely turn to external sources. Influential staff said they would rely equally on internal and external sources. These information pathways should be further monitored to assess whether how messages are likely to successfully reach each group.



Documents

Market Progress Evalu ation Report Efficient Building ... · Market Progress Evalu ation Report Efficient Building Practices Initiative,No.2 ... Response to the Website Usability