LEARNING FROM OUR BUILDINGS - Egypt Arch from... · 2011-03-15 · LEARNING FROM OUR BUILDINGS A State-of-the-Practice Summary of Post-Occupancy Evaluation Federal Facilities Council

LEARNING FROM OUR BUILDINGSA State-of-the-Practice Summary of

Post-Occupancy Evaluation

Federal Facilities Council Technical Report No. 145

NATIONAL ACADEMY PRESSWashington, D.C.

NATIONAL ACADEMY PRESS 2101 Constitution Avenue, N.W. Washington, DC 20418

NOTICE

The Federal Facilities Council (FFC) is a continuing activity of the Board on Infrastructure and theConstructed Environment of the National Research Council (NRC). The purpose of the FFC is topromote continuing cooperation among the sponsoring federal agencies and between the agenciesand other elements of the building community in order to advance building science and technol-ogy—particularly with regard to the design, construction, acquisition, evaluation, and operation offederal facilities. The following agencies sponsor the FFC:

Department of the Air Force, Office of the Civil EngineerDepartment of the Air Force, Air National GuardDepartment of the Army, Assistant Chief of Staff for Installation ManagementDepartment of Defense, Federal Facilities DirectorateDepartment of EnergyDepartment of the Interior, Office of Managing Risk and Public SafetyDepartment of the Navy, Naval Facilities Engineering CommandDepartment of State, Office of Overseas Buildings OperationsDepartment of Transportation, U.S. Coast GuardDepartment of Transportation, Federal Aviation AdministrationDepartment of Veterans Affairs, Office of Facilities ManagementFood and Drug AdministrationGeneral Services Administration, Public Buildings ServiceIndian Health ServiceInternational Broadcasting BureauNational Aeronautics and Space Administration, Facilities Engineering DivisionNational Institute of Standards and Technology, Building and Fire Research LaboratoryNational Institutes of HealthNational Science FoundationSmithsonian Institution, Facilities Engineering and OperationsU.S. Postal Service, Engineering Division

As part of its activities, the FFC periodically publishes reports that have been prepared by committeesof government employees. Because these committees are not appointed by the NRC, they do notmake recommendations, and their reports are considered FFC publications rather than NRC publi-cations.

For additional information on the FFC program and its reports, visit the Web site at <http://www4.nationalacademies.org/cets/ffc.nsf> or write to Director, Federal Facilities Council, 2101Constitution Avenue, N.W., HA-274, Washington, DC 20418 or call 202-334-3374.

Printed in the United States of America

2001

iii

FEDERAL FACILITIES COUNCIL

Chair

Henry J. Hatch, U.S. Army (Retired)

Vice Chair

William Brubaker, Director, Facilities Engineering and Operations, Smithsonian Institution

Members

Walter Borys, Operations and Maintenance Division, International Broadcasting BureauJohn Bower, MILCON Program Manager, U.S. Air ForcePeter Chang, Division of Civil and Mechanical Systems, National Science FoundationTony Clifford, Director, Division of Engineering Services, National Institutes of HealthJose Cuzmé, Director, Division of Facilities Planning and Construction, Indian Health ServiceDavid Eakin, Chief Engineer, Office of the Chief Architect, Public Buildings Service, General Services

AdministrationJames Hill, Deputy Director, Building and Fire Research Laboratory, National Institute of Standards and TechnologyJohn Irby, Director, Federal Facilities Directorate, U.S. Department of DefenseL. Michael Kaas, Director, Office of Managing Risk and Public Safety, U.S. Department of the InteriorJoe McCarty, Engineering Team Leader, U.S. Army Corps of EngineersWilliam Miner, Acting Director, Building Design and Engineering, Office of Overseas Buildings Operations,

U.S. Department of StateWilliam Morrison, Chief, Structures Branch, Facilities Division, Air National GuardGet Moy, Chief Engineer and Director, Planning and Engineering Support, Naval Facilities Engineering

Command, U.S. NavyRobert Neary, Jr., Associate Facilities Management Officer, Office of Facilities Management, U.S.

Department of Veterans AffairsJuaida Norell, Airways Support Division, Federal Aviation AdministrationWade Raines, Maintenance and Policies Programs, Engineering Division, U.S. Postal ServiceJames Rispoli, Director, Engineering and Construction Management Office, U.S. Department of EnergyWilliam Stamper, Senior Program Manager, Facilities Engineering Division, National Aeronautics and Space

AdministrationStan Walker, Division Chief, Shore Facilities Capital Asset Management, U.S. Coast Guard

Staff

Richard Little, Director, Board on Infrastructure and the Constructed Environment (BICE)Lynda Stanley, Director, Federal Facilities CouncilMichael Cohn, Program Officer, BICEKimberly Goldberg, Administrative Associate, BICENicole Longshore, Project Assistant, BICE

v

At the most fundamental level, the purpose of abuilding is to provide shelter for activities that couldnot be carried out as effectively, or carried out at all, inthe natural environment. Buildings are designed andconstructed to (1) protect people and equipment fromelements such as wind, rain, snow, and heat; (2) pro-vide interior space whose configuration, furnishings,and environment (temperature, humidity, noise, light,air quality, materials) are suited to the activities thattake place within; and (3) provide the infrastructure—water, electricity, waste disposal systems, fire suppres-sion—necessary to carry out activities in a safe envi-ronment.

Today, people and organizations have even higherexpectations for buildings. Owners expect that theirinvestments will result in buildings that support theirbusiness lines or missions by enhancing worker pro-ductivity, profits, and image; that are sustainable,accessible, adaptable to new uses, energy efficient, andcost-effective to build and to maintain; and that meetthe needs of their clients. Users expect that buildingswill be functional, comfortable, and safe and will notimpair their health. A building’s performance is itscapacity to meet any or all of these expectations.

Post-occupancy evaluation (POE) is a process forevaluating a building’s performance once it is occu-pied. It is based on the idea that finding out about users’needs by systematically assessing human response tobuildings and other designed spaces is a legitimate aimof building research. Early efforts at POE focused onhousing needs of disadvantaged groups to improveenvironmental quality in government-subsidized hous-ing. The process was later applied to other governmentfacilities such as military housing, hospitals, prisons,

Preface

and courthouses. POE began to be used for office build-ings and other commercial real estate in the mid-1980sand continues to be used for a variety of facility typestoday.

As POE has been applied to a larger range of build-ing types and as expectations for buildings haveevolved, POE has come to mean any and all activitiesthat originate out of an interest in learning how a build-ing performs once it is built, including whether andhow well it has met expectations and how satisfiedbuilding users are with the environment that has beencreated. Although POEs are still focused on determin-ing user comfort and satisfaction, organizations areattempting to find ways to use the information gatheredto support more informed decision-making about spaceand building investments during the programming,design, construction, and operation phases of afacility’s life cycle. To do this, organizations need toestablish design criteria, databases or other methodsfor compiling lessons from POEs and for disseminat-ing those lessons throughout the organization, fromsenior executives to midlevel managers, project man-agers, consultants, and clients.

The federal government is the United States’ largestowner of facilities, with approximately 500,000 facili-ties worldwide. Federal agencies that own, use, or pro-vide facilities have a significant interest in optimizingtheir performance. The General Services Administra-tion, U.S. Army Corps of Engineers, Naval FacilitiesEngineering Command, U.S. Postal Service, StateDepartment, and Administrative Office of the U.S.Courts have been leaders in the development and prac-tice of POEs. They and other federal agencies are try-ing to find ways to share information about effective

vi PREFACE

processes for conducting POEs, to capture and dissemi-nate lessons learned, and to increase the value thatPOEs add to the facility acquisition process.

The Federal Facilities Council (FFC) is a coopera-tive association of 21 federal agencies with interestsand responsibilities for large inventories of buildings.The FFC is a continuing activity of the Board on Infra-structure and the Constructed Environment of theNational Research Council (NRC), the principal oper-ating agency of the National Academy of Sciences andthe National Academy of Engineering. In 1986, theFFC requested that the NRC appoint a committee toexamine the field and propose ways by which the POEprocess could be improved to better serve public andprivate sector organizations. The resulting report, Post-Occupancy Evaluation Practices in the BuildingProcess: Opportunities for Improvement, proposed abroader view of POEs—from being simply the endphase of a building project to being an integral part ofthe entire building process. The authoring committeerecommended a series of actions related to policy, pro-cedures, and innovative technologies and techniques toachieve that broader view.

In 2000, the FFC funded a second study to look atthe state of the practice of POEs and lessons-learnedprograms among federal agencies and in private, pub-lic, and academic organizations both here and abroad.The sponsor agencies specifically wanted to determinewhether and how information gathered during POE

processes could be used to help inform decisions madein the programming, budgeting, design, construction,and operation phases of facility acquisition in a usefuland timely way. To complete this study, the FFC com-missioned a set of papers by recognized experts in thisfield, conducted a survey of selected federal agencieswith POE programs, and held a forum at the NationalAcademy of Sciences on March 13, 2001, to addressthese issues. This report is the result of those efforts.

Within the context of a rapidly changing buildingindustry and the introduction of new specialty fieldsand technologies into the building process and newdesign objectives for buildings that are sustainable,healthy, and productivity enhancing, and with ever-greater demands on limited resources, POE constitutesa potentially vital contribution in the effort to maintainquality assurance. Within the federal government, thedownsizing of in-house facilities engineering organi-zations, the increased outsourcing of design and con-struction functions, and the loss of in-house technicalexpertise, all underscore the need for a strong capabil-ity to capture and disseminate lessons learned as part ofa dynamic project delivery process. We hope this reportwill help federal agencies and other organizations toenhance those capabilities.

Lynda StanleyDirector, Federal Facilities Council

vii

Contents

1 OVERVIEW: A SUMMARY OF FINDINGS 1Introduction, 1Organization of This Report, 1Summary of Findings, 2References, 8

2 THE EVOLUTION OF POST-OCCUPANCY EVALUATION: TOWARD BUILDINGPERFORMANCE AND UNIVERSAL DESIGN EVALUATION 9Wolfgang Preiser, Ph.D., University of CincinnatiPost-Occupancy Evaluation: An Overview, 9Types of Evaluation for Building Projects, 10Purposes of POEs, 11Types of POEs, 11Benefits, Uses, and Costs of POEs, 12An Integrative Framework for Building Performance Evaluations, 13Universal Design Evaluation, 15Universal Design Performance, 15Performance Levels, 16Toward Universal Design Evaluation, 19Possible Strategies for Universal Design Evaluation, 20Education and Training in Universal Design Evaluation Techniques, 21Conclusions, 21About the Author, 21References, 22

3 POST-OCCUPANCY EVALUATION: A MULTIFACETED TOOL FOR BUILDINGIMPROVEMENT 23Jacqueline Vischer, Ph.D., University of MontrealWhat Is Post-Occupancy Evaluation?, 23The Pros and Cons of POE, 23Current Status of POE, 24Best Practices, 29Managing POE Information, 30The Future of POE: Recommendations for an Unobtrusive POE Process , 32

viii CONTENTS

About the Author, 33References, 34

4 POST-OCCUPANCY EVALUATION PROCESSES IN SIX FEDERAL AGENCIES 35Survey Questions, 35Summary of Findings, 36Descriptions of POE Programs, 37

5 POST-OCCUPANCY EVALUATIONS AND ORGANIZATIONAL LEARNING 42Craig Zimring, Ph.D., Georgia Institute of Technology, andThierry Rosenheck, Office of Overseas Buildings Operations, U.S. Department of StateBrief Introduction to Post-Occupancy Evaluation, 44Do Organizations Do POE-Enabled Organizational Learning?, 45Ways to Create the Appropriate Conditions for Learning Through POE, 46Creating a Knowledge Base for Building Delivery and Management, 49Building on Existing Evaluations, 50Lessons from POE Programs: Enhancing Organizational Learning, 51About the Authors, 52References, 52

6 THE ROLE OF TECHNOLOGY FOR BUILDING PERFORMANCE ASSESSMENTS 54Audrey Kaplan, Workplace Diagnostics Ltd.Introduction, 54Cybersurveys, 55Response Rate, 56Sampling, 56Lessons Learned, 56Conclusions and Discussion, 58About the Author, 59References, 59

APPENDIXES

A FUNCTIONALITY AND SERVICEABILITY STANDARDS: TOOLS FOR STATINGFUNCTIONAL REQUIREMENTS AND FOR EVALUATING FACILITIES 63Francoise Szigeti and Gerald Davis, International Centre for Facilities

B A BALANCED SCORECARD APPROACH TO POST-OCCUPANCY EVALUATION: USINGTHE TOOLS OF BUSINESS TO EVALUATE FACILITIES 79Judith Heerwagen, Ph.D., J.H. Heerwagen and Associates

C Supplemental Information to Chapter 3 88

D Supplemental Information to Chapter 4 95

E Supplemental Information to Chapter 6 116

F Chapter 5 from Post-Occupancy Evaluation Practices in the Building Process:Opportunities for Improvement 119

BIBLIOGRAPHY 126

1

1

Overview: A Summary of Findings

INTRODUCTION

Post-occupancy evaluation (POE) is a process ofsystematically evaluating the performance of buildingsafter they have been built and occupied for some time.POE differs from other evaluations of building perfor-mance in that it focuses on the requirements of buildingoccupants, including health, safety, security, function-ality and efficiency, psychological comfort, aestheticquality, and satisfaction. “Lessons learned” refers toprograms aimed at collecting, archiving, and sharinginformation about successes and failures in processes,products, and other building-related areas for the pur-pose of improving the quality and life-cycle cost offuture buildings. Ideally, the information gainedthrough POEs is captured in lessons-learned programsand used in the planning, programming, and designprocesses for new facilities to build on successes andavoid repeating mistakes.

In 2000 the Federal Facilities Council, a cooperativeassociation of 21 federal agencies with interests andresponsibilities for large inventories of buildings,funded a study to look at the state of the practice ofPOEs and lessons-learned programs in federal agen-cies and in private, public, and academic organizationsboth in the United States and abroad. The primary pur-pose was to produce a report that identified successfulpost-occupancy evaluation programs (those that haveworked well in terms of impact, longevity, and usersatisfaction) and lessons-learned programs in federalagencies and the private sector. Specific objectiveswere to identify:

• an industry-accepted definition of POEs;• methods and technologies used for data collection;

• the costs of POE surveys;• the benefits of conducting POEs and capturing

lessons;• organizational barriers to conducting POEs;• a standardized methodology that could be used

within agencies to assure consistency in data gath-ering and allow for cooperative development ofbenchmarks and best practices; and

• performance measures for POE programs.

To produce this report the Federal Facilities Councilcommissioned a set of papers by recognized subjectmatter experts, conducted a survey of six federal agen-cies with POE programs, and held a forum at theNational Academy of Sciences on March 13, 2001.

ORGANIZATION OF THIS REPORT

` The next sections of Chapter 1 summarize the find-ings contained in the authored papers, the survey ques-tionnaires, and the forum presentations as they relate tothe study objectives. In Chapter 2, “The Evolution ofPost-Occupancy Evaluation: Toward Building Perfor-mance and Universal Design Evaluation,” WolfgangPreiser reviews the historical development of POEprograms, their uses, costs, and benefits; describes anintegrative framework for building performance; andoutlines a new paradigm for universal design evalua-tion. Chapter 3, “Post-Occupancy Evaluation: A Multi-faceted Tool for Building Improvement,” written byJacqueline Vischer, discusses the historical basis forPOE programs; identifies the discrepancy that existsbetween reasons for doing POEs and the difficulties ofimplementing them; describes successful POE pro-

2 LEARNING FROM OUR BUILDINGS

grams employing the building-in-use assessment sys-tem; and makes recommendations for an unobtrusivePOE process. Chapter 4, “Post-Occupancy EvaluationPrograms in Six Federal Agencies,” summarizes thesurvey questionnaire findings and describes current andemerging POE practices in those agencies. In Chap-ter 5, “Post-Occupancy Evaluations and OrganizationalLearning,” Craig Zimring and Thierry Rosenheck iden-tify the elements necessary for organizational learning;explore how 18 organizations have used POEs success-fully for organizational learning; and discuss thelessons-learned role of POEs. Chapter 6, “The Role ofTechnology for Building Performance Assessments,”authored by Audrey Kaplan, identifies technologiesthat have been used for POE processes; explores thepossibilities of cybersurveys for improving the responserate and lowering the costs of POE data collection andanalysis; and discusses Web survey design consider-ations, sampling techniques, publicity, data collection,and responses.

In Appendix A, “Functionality and Standards: Toolsfor Stating Functional Requirements and for Evaluat-ing Facilities,” Francoise Szigeti and Gerald Davis dis-cuss how the ASTM Standards on Whole BuildingFunctionality and Serviceability (ASTM, 2000) can beused to evaluate the quality of the performance deliv-ered by a facility in relation to the original expecta-tions. Appendix B, “A Balanced Scorecard Approachfor Post-Occupancy Evaluation: Using the Tools ofBusiness to Evaluate Facilities,” written by JudithHeerwagen, outlines a performance-based approachthat could provide an analytical structure to the entireprocess, from original concept through lessons learned.Appendixes C-F contain supporting materials. Thebibliography is a compilation of references cited in thetext and additional references submitted by the authors.

SUMMARY OF FINDINGS

Post-occupancy evaluation is based on the idea thatbetter living space can be designed by asking usersabout their needs. POE efforts in Britain, France,Canada, and the United States in the 1960s and 1970sinvolved individual case studies focusing on buildingsaccessible to academic researchers, such as publichousing and college dormitories. Information fromoccupants about their response to buildings wasgathered through questionnaires, interviews, site visits,and observation; sometimes the information was linked

to the physical assessment of a building. The lessonsfrom these studies were intended to convey what designelements work well, what works best, and what shouldnot be repeated in future buildings.

POE efforts in the United States and abroad wereprimarily focused on government and other publicbuildings from the 1960s to the mid-1980s. Private sec-tor organizations in the United States became moreactively involved with POE after the release of UsingOffice Design to Increase Productivity (Brill et al.,1985), which linked features of the office environmentwith worker productivity. As corporations downsizedand reengineered their business processes to focus oncore competencies, chief executive officers began tothink of their buildings as ways to achieve such strate-gic goals as customer satisfaction, decreased time tomarket, increased innovation, attraction and retentionof high-quality workers, and enhanced productivity ofwork groups. A number of organizations have sinceused POE as a tool for improving, innovating, or other-wise initiating strategic workspace changes.

Industry-Accepted Definition

As POEs have become broader in scope and purpose,POE has come to mean any activity that originates outof an interest in learning how a building performs onceit is built (if and how well it has met expectations) andhow satisfied building users are with the environmentthat has been created. POE has been seen as one of anumber of practices aimed at understanding designcriteria, predicting the effectiveness of emergingdesigns, reviewing completed designs, supportingbuilding activation and facilities management, andlinking user response to the performance of buildings.POE is also evolving toward more process-orientedevaluations for planning, programming, and capitalasset management.

As a consequence, there is no industry-accepteddefinition for POE; nor is there a standardized methodfor conducting a POE. Even the term POE has comeunder scrutiny. Academics and others working in thefield have proposed new terms, including environ-mental design evaluations, environmental audits, build-ing-in-use assessments, building evaluation, facilityassessment, post-construction evaluation, and buildingperformance evaluations in an effort to better reflectthe objectives and goals of POEs as they are practiced.

OVERVIEW: A SUMMARY OF FINDINGS 3

Methods and Technologies for Data Collection

Traditionally, POEs are conducted using question-naires, interviews, site visits, and observation of build-ing users. Over time, more specific processes, levels ofsurveys, and new technologies have been developed tobetter fit stakeholders’ objectives and budgets. Shortcutmethods have been devised that allow the researcher orevaluator to obtain valid and useful information in lesstime than previously.

Use of the Web and other technologies could sub-stantially change the methods for conducting POEs andfor analyzing the data generated. Web-based cyber-surveys may become the primary survey instrument,owing to their convenience, low cost of distributionand return, ability to check for errors and receive data—including rich-text replies—in electronic format, andease with which respondents can receive feedback.Two U.S. federal agencies have already begun movingin this direction. The Public Buildings Service of theGeneral Services Administration is working with theCenter for the Built Environment at the University ofCalifornia, Berkeley, to develop a set of POE surveysthat can be administered over the Web. Different sur-veys are directed to different key personnel to helpdetermine if GSA is meeting a number of key manage-ment indicators. The Naval Facilities EngineeringCommand is modifying its database to integrate corpo-rate management systems and to Web-enable its POEsurvey. The POE survey will draw information fromthe management system and alert individuals when thesurveys should be administered.

For organizations seeking to link facility design andbusiness goals, a POE approach could be used thatcombines assessment of the physical condition of thebuilding and its systems with assessment of user com-fort on such topics as indoor air quality and ventilationrates, lighting levels and contrast conditions, building(not occupant) noise levels, and indoor temperature(thermal comfort). Results from subjective or instru-ment measures could be plotted on floor plans usinggeographical information systems. The data could thenbe analyzed individually or as overlays showing thespatial distribution of a range of factors. For example,ratings of thermal comfort could be assessed with tem-perature data and spatial location. Occupants’ percep-tions of interior environments could also be linked withdata from building control systems, local weather con-ditions, or facility usage as recorded by building-accesssmart cards. The Disney Corporation and the World

Bank both have linked POE data to their geographicinformation systems for future planning and designpurposes.

Costs of Post-Occupancy Evaluation Surveys

Depending on the type of survey conducted and thelevel of analysis used, the cost for a POE survey canrange from a few thousand dollars per facility to U.S.$2.50 or more per square foot of space evaluated. Fed-eral agencies have reported costs ranging from $1,800for a simple standard questionnaire that could be com-pleted in one hour to $90,000 for an in-depth analysis,including several days of interviews, the use of multi-disciplinary teams, site visits, and report writing. Todaythe range of methods for conducting POEs allows anorganization to tailor the technique to its objectives andavailable resources (time, staff, and money). Web-enabled surveys are emerging, and these may provideanother technique that can be used at a relatively lowcost.

Benefits of Conducting Post-Occupancy Evaluationsand Capturing Lessons

Stakeholders in buildings include investors, owners,operators, designers, contractors, maintenance person-nel, and users or occupants. A POE process that cap-tures lessons can serve many purposes and providemany benefits, depending on a stakeholder’s goals andobjectives. These include the following:

• support of policy development as reflected indesign and planning guides. The validity ofunderlying premises used in recurrent designs canbe tested and evolutionary improvements to pro-gramming and design criteria can be identifiedand incorporated into standards and guidanceliterature.

• provision to the building industry of informationabout buildings in use by improving the measure-ment of building performance by quantifyingoccupant perceptions and physical environmentalfactors.

• testing of new concepts to determine how wellthey work in occupied buildings.

• generation of information needed to justify majorexpenditures and inform future decisions. Infor-mation generated by POEs can be used fordecision-making in the pre-design phase of a new


project to avoid repeating past mistakes. It canalso be used to educate decision makers about theperformance implications of design changes dic-tated by budget cuts and to improve the way spaceis used as determined by stakeholders or docu-mented standards.

• improvement of building performance through-out the life cycle. POEs can be used to identifyand remediate such problems associated with newbuildings as uncontrolled leakage, deficient aircirculation, poor signage, and lack of storage. Forfacilities that incorporate the concept of adapt-ability, where changes are frequently necessary,regularly conducted POEs can contribute to anongoing process of adapting the facility to chang-ing organizational needs.

• making design professionals and owners account-able for building performance. POEs can be usedto measure the functionality and appropriatenessof a design and establish conformance withexplicit and comprehensive performance require-ments as stated in the functional program. Theycan also serve as a mechanism to monitor abuilding’s quality and to notify decision makerswhen the performance of a building does notreach an agreed standard.

• aiding communications among stakeholders suchas designers, clients, facility managers, and endusers. Through active involvement in the evalua-tion process, the attitude of building occupantscan be improved and proactive facility manage-ment that responds to building users’ values canbe facilitated.

Barriers to Conducting Post-Occupancy Evaluations

Despite these benefits, only a limited number oflarge organizations and institutions have active POEprograms. Relatively few organizations have fullyincorporated lessons from POE programs into theirbuilding delivery processes, job descriptions, or report-ing arrangements. One reason for this limited use is thenature of POE itself, which identifies both successesand failures. Most organizations do not reward staff orprograms for exposing shortcomings. In addition, rela-tively few organizations have created appropriate con-ditions for learning (i.e., conditions that allow theorganization to constantly improve the way it operatesunder routine conditions and to respond to changequickly and effectively when the need arises).

Additional barriers to more effective use of POE andlessons-learned programs include the following:

• the difficulty of establishing a clear causal linkbetween positive outcomes and the physical envi-ronment. This lack of a clear link can make it dif-ficult for POE proponents to convince decisionmakers that the benefits received will justify theexpenditure of time and money on the evalua-tions.

• reluctance by organizations and building profes-sionals to participate in a process that may exposeproblems or failures or may be used as a methodto focus (or deflect) blame. For federal agencies,senior executives may be concerned that identify-ing problems may be considered a weakness byCongress or the inspector general.

• fear of soliciting feedback from occupants on thegrounds that both seeking and receiving this typeof information may obligate an organization tomake costly changes to its services or to the build-ing itself.

• lack of participation by building users. In somecases the reluctance to participate can be attrib-uted directly to uncertainty about senior manage-ment’s commitment to the program, which maybe manifested by lack of resources or visibleendorsement of the program.

• failure to distribute information resulting fromPOEs to decision makers and other stakeholders.

• pressure to meet design and construction dead-lines, which can create a time barrier to sustainedPOE activity. Staffs may be so focused on futureprojects and ongoing construction that POEs forcompleted and occupied projects are given lowerpriority.

• lack of in-house staff having the wide range ofskills and technical expertise needed to direct andmanage the results of evaluations and to commu-nicate the information so that it is useful and non-threatening. Organizations may be reluctant tohire consultants to conduct and analyze POEs ifresources are limited and there is a lack ofexecutive-level commitment to such programs.For federal agencies it may be difficult to obtainor earmark the funding needed to conduct POEsregardless of whether they are using consultantsor in-house staff.

• organizational structures can create barriers whenresponsibilities for POE administration and


lessons-learned database development are assignedto different offices, thereby creating a need forinteroffice collaboration and blurring the lines ofaccountability.

Successful Post-Occupancy Evaluation and Lessons-Learned Programs

Despite the barriers mentioned above, POE has con-tinued to grow as a practice. Some organizations havebeen able to effectively integrate the lessons of POEsinto strategic planning and decision-making processesfor facility delivery and acquisition. Notable examplesinclude the following:

• the U.S. Army Corps of Engineers program ofparticipatory programming and evaluation duringthe 1970s: The effort, undertaken after researchindicated that aging facilities were an impedimentto recruiting and retaining soldiers for the all-volunteer Army, resulted in design guides forfacilities ranging from drama and music centersto barracks and military police stations.

• the U.S. Postal Service program: In the 1980s thenewly reorganized U.S. Postal Service madeextensive use of POEs to gather information aboutbuildings to develop retail-focused postal storesto better compete with private sector companies.The program proved successful in meeting itsobjectives. Over time the survey methodologieshave been modified to support new objectives,but the Postal Service program remains active.

• the Disney Corporation, which uses three evalua-tion programs and corresponding databases toexplore optimal conditions and develop predic-tors of its key business driver, intention of thecustomer to return: The databases are used exten-sively in the design and renovation of buildingsbecause they allow Disney to make direct linksbetween inputs, such as proposed number ofpeople entering gates, and outputs, such as thewidth of Main Street.

• the Massachusetts Division of Capital Planningand Operations, which links POE with pre-designprogramming of public buildings: POEs havebeen used to develop and test prototype conceptsfor state police stations, child care centers, andvehicle maintenance and repair stations, resultingin savings of cost and time in the programming,design, and construction of new facilities.

• Bell Canada and the World Bank: Both organiza-tions have attempted to use POE as an asset man-agement tool for space planning. Both companiescollected large amounts of data from occupantsand calculated baseline scores on seven comfortfactors across all buildings. Real estate staff couldthen identify which spaces exceeded baselinescores and which fell below. These factors areconsidered in budgeting for maintenance andrepair, space planning, and the reconfiguration ofspace. The World Bank also made an effort tolink its database with computer-aided designdrawings so that baseline scores of buildings, orfloors of buildings, could be used as indexes ofquality in its space-planning process.

Elements for Post-Occupancy Evaluation Success

One objective of this study was to identify a stan-dardized methodology that could be used by federalagencies to assure consistency in data gathering andallow for cooperative development of benchmarks andbest practices. As outlined above, POEs can serve avariety of purposes and the methods used for POE sur-veys can be tailored to the specific purpose and avail-able resources. It is not clear that a standardized meth-odology for POEs that could be used for benchmarkingacross federal agencies would be effective or evendesirable. However, based on the information in thefollowing chapters, it is evident that organizations seek-ing to establish or restructure POE programs need tomake a number of key decisions in the early planningstages and develop and incorporate several key com-ponents in their program if they are to be successful,regardless of the POE purpose or methodology. Thesedecisions and components are identified below.

• Develop a clear statement about what the organi-zation wants to achieve by conducting and apply-ing POEs. The links between evaluations andstated requirements have to be explicit and easyto trace.

• Identify the resources available to carry out thePOE, matching data collection and analysisactivities to the available time and budget.

• Identify the likely users of POE results and deter-mine how they need the results communicated.

• Garner support from high levels of the organiza-tion to signal the importance of the project or pro-gram to people throughout the organization.


• Determine if it is to be a one-of-a-kind case studyor a standardized approach that allows buildingprofessionals to collect modest amounts of com-parable data from a variety of buildings on a com-parative basis over time.

• For a standard approach, capture and present theinformation that is fed forward from such activi-ties in comparable formats. Develop acceptedterminology, standard definitions, and normalizeddocumentation at the outset to make such com-parisons easier.

• Have the questionnaire designed and analyzed bysomeone skilled in survey research. Determinewhat indicators will be selected for measurement.Evaluate such measures for usefulness, reliability,validity (they should measure what they purportto measure), efficiency, ability to allow smallchanges to be noticed, and balance. The entire setof measures should include both quantitative andqualitative measures, as well as direct and indi-rect measures.

• Consider POE techniques that avoid direct ques-tioning of users, for example, using data gener-ated by building control systems, observations,and expert walk-throughs. The evaluation processwill fail if occupants are reluctant to participateor if staff resources are insufficient to help withthe organizational data gathering or for othermeasures.

• Decide whether user survey data will be madeavailable to building occupants and, if so, in howmuch detail and for what purpose.

• Inform facility occupants about the purpose oftheir involvement in providing feedback and howthe data will be used. They need to be assured thattheir own input will be kept confidential andshould be informed if immediate correction ofproblems is not envisioned. Occupants are morelikely to be engaged in the process if they areinvolved in helping design the measurement planand if they see a benefit from participation.

Elements for Successful Lessons-Learned Programs

Issues related to applying lessons learned from POEsare only partly technical. Tools for creating Web sitesand databases are now widely available and relativelyinexpensive. Optimizing the value of POEs requiresthe initiative to collect the information, the time tomake sense of it, and the will to share it. To success-

fully incorporate the lessons from POEs into capitalasset management processes and decision making,organizations need to do the following:

• Gain the support and long-term commitment ofsenior management.

• Create broad opportunities for participation andreflection. Encourage the direct participation ofbuilding owners, decision makers, designers,customers, project managers, and staff in theevaluations. Where necessary, create incentivesfor participation, for instance, through contractclauses, “free” vacation days, or using POEresults as part of a review of qualifications whenselecting consultants and contractors.

• Provide access to the information for differentaudiences. Upper-level management, architects,engineers, project managers, clients, and buildingusers have different levels of responsibility in thebuilding process and require different informa-tion from POEs. Lessons learned through POEsshould be presented in a variety of formats to meetthe needs of various stakeholders; these formatscan include databases, design guides, case studyreports, and policy and planning documents.

• Create simple databases that can be accessed bykey words and that use simple methods to displayoverall results to aid interpretation. Ideally, adatabase should include the design hypothesesand assumptions for each project, the specificmeasures used to test the hypothesis, before andafter photos of the space, brief summaries of thedata, some analyses of cost, size and materials,key lessons learned, connections to other studies,connections to the full research findings beforeand after, and recommendations for future designs.

• Identify the critical points in the building processat which information or a POE can help resolve aproblem or issue of considerable importance toparticipants. Use the information gathered todevelop or modify policy documents.

• Build on facility evaluations that are the subjectof complaints or controversy. Focus on under-standing the reasons for problems or failures, anduse this information to modify programming orplanning processes to avoid repeating the experi-ence.

• Use POEs of innovative buildings or componentsto help decide whether such innovations shouldbe considered for future buildings.


• Create protected opportunities for innovation andevaluation. Methods for doing this include sanc-tioning research with the clear understanding thatnot all innovations will be successful, startingsmall with projects that have an experimentalcomponent, and evaluating the results beforeapplying them on a broader basis.

Performance Measures

The use of performance-based approaches to facilityacquisition and evaluation is a worldwide trend.Performance-based approaches require greater atten-tion to the definition and description of purposes(demand and results) of a project or program, both inthe short and long term, and to ways of measuringwhether the desired results have been achieved.

Performance criteria for POEs for individual build-ings are based typically on the stated design intent andcriteria contained in or inferred from a functionalprogram. Measures include indicators related to orga-nizational and occupant performance, such as workersatisfaction and productivity, and safety and security,but may also include measures of building performanceas perceived by users such as air quality, thermal com-fort, spatial comfort, ergonomics, privacy, lightingcomfort, noise (from the building and offices), andaesthetics.

A performance-based approach that could be usedto measure the quality of services delivered by thefacility in support of individuals or groups involves theuse of scales created by the International Centre forFacilities. The scales have been approved and publishedby the American Society for Testing and Materials(ASTM) as ASTM Standards on Whole Building Func-tionality and Serviceability (ASTM, 2000). The ASTMstandard scales include two matched, multiple-choicequestionnaires. One questionnaire, used for settingworkplace requirements for functionality and quality,describes customer needs (demand) as the core of front-end planning. The other, matching questionnaire isused for assessing the capability of a building or designto meet these levels of need, which represents itsserviceability. It rates facilities (supply) in terms of per-formance as a first step toward an outline performancespecification. In the pre-project planning phase, thescales can be used to determine if the proposed facilitywill meet the needs of the customer and, if not, thechanges that can be made to improve the fit. Once the

facility is built a second evaluation can be made todetermine how closely the facility fits the originalexpectations.

Evaluating the performance of buildings as a finan-cial asset within a portfolio or inventory of buildings ismore difficult. One potential approach is the BalancedScorecard system of measurement adapted to facilities.The Balanced Scorecard is a business tool that assessesfour categories of performance: financial, businessprocess, customer relations, and learning and growth(human resource development). The four categories areused to avoid overemphasis on financial incomes, tocapture the full value of the product or process, and tobalance levels of analysis from individual and groupoutcomes to higher-level organizational outcomes.Performance measures for facilities often overempha-size costs because there are few data to show linkagesbetween facility design and business goals. Cost-focused strategies include reducing the size of workstations, using a universal plan with only a few workstation options, eliminating private offices or person-ally assigned spaces, and telecommuting. A BalancedScorecard approach, adapted to facilities, can answerquestions such as the following:

• How can workplace design positively influenceoutcomes that organizations value?

• How can workplace design reduce costs orincrease revenue?

• How can workplace design enhance humanresource development?

• How can the physical workplace enhance workprocesses and reduce time to market?

• How can the physical work environment enhancecustomer relationships and present a more posi-tive face to the public?

By asking these questions at the beginning of adesign project the Balanced Scorecard approach canprovide an analytical structure to the entire process,from conceptualization through evaluation and finallyto lessons learned. POE results can be used to developmeasures for all categories of performance and toevaluate the organization’s success in meeting its per-formance goals. A core set of measures can be usedacross facilities to gain a better understanding of theentire building stock, while other measures would beunique to the goals and objectives of the particularorganization, department, or division.


POEs and Lessons-Learned Programs for FederalAgencies

Federal agencies that own, provide, use or managelarge inventories of facilities face a number of chal-lenges. They are responsible for delivering buildingsthat are safe, secure, sustainable, accessible, cost effec-tive to operate and maintain, responsive to customerneeds, and supportive of worker productivity. Agen-cies must also become more business-like in their prac-tices, more accountable to the public, and to managetheir processes such that the results are measurable (i.e.,on time, within budget, and of a quality to last 50 yearsor longer). In most agencies, the staff resources to meetthese challenges were reduced during the 1990s andagencies face further loss of technical expertise throughretirements and attrition.

In this environment, POE and lessons-learned pro-grams, appropriately designed and managed, can addsignificant value to federal facility acquisition pro-cesses. A constructed facility is a culmination of poli-cies, actions, and expenditures that call for evaluation.POE and lessons-learned programs can provide a sys-tematic method for assessing the impact of past deci-sions and using these assessments in future decisionmaking. They can partially mitigate the loss of in-housestaff by creating an institutional data base that remainswith the agency through changes in management andattrition in personnel. They also provide an opportu-nity to improve user satisfaction and reduce the cost ofdesign development by sharing information through-out an agency and with outside contractors.

Although POE and lessons-learned programs havebeen instituted in relatively few federal agencies, thoseagencies have reported significant benefits. The GeneralServices Administration, the Administrative Office ofthe U.S. Courts, and the U.S. State Department, among

others, have used lessons from POE surveys to improvethe design of federal buildings, to lower operating andmaintenance costs, and to provide quality assurance.The Naval Facilities Engineering Command and theArmy Corps of Engineers, among others, have usedPOE data to design buildings that better meet userneeds and help to support the retention of military andcivilian staff. The U.S. Postal Service has used POEdata to better meet its customer needs and, in so doing,the USPS has been better able to compete with privatesector mail delivery services. The U.S. Air Force andothers have used POE data for quality assurance inidentifying building defects and repairing them earlyin the life of the building when it is most cost-effectiveto make such repairs. The GSA and NAVFAC arerestructuring their programs to better link POE to awide range of facility management processes as federalagencies shift their emphasis from facility engineeringto capital asset management.

As Web-based surveys, building controls, and geo-graphic information systems continue to evolve, con-ducting POE programs and capturing lessons shouldbecome easier and yield more useful data. Institutingthese programs in additional federal agencies in accordwith the guidelines highlighted above should result inbenefits that outweigh the costs. However, establish-ment of these programs will require leadership fromboth senior executives and program managers and awillingness to learn from both successes and failures.

REFERENCESASTM (American Society for Testing and Materials). 2000. ASTM Stan-

dards on Whole Building Functionality and Serviceability. WestConshohocken, Pa.: ASTM.

Brill, M., S.M., Margulis, and E., Konar, 1985. Using Office Design toIncrease Productivity (2 vols.). Buffalo, N.Y.: BOSTI andWestinghouse Furniture Systems.

9

2

The Evolution of Post-Occupancy Evaluation:Toward Building Performance and

Universal Design EvaluationWolfgang F.E. Preiser, Ph.D., University of Cincinnati

The purpose of this chapter is to define and providea rationale for the existence of building performanceevaluation. Its history and evolution from post-occupancyevaluation over the past 30 years is highlighted. Majormethods used in performance evaluations are presentedand the estimated cost and benefits described. Training,opportunities and approaches for building performanceevaluation are enumerated. Possible opportunities forgovernment involvement in building performanceevaluation are sketched out. The next step and newparadigm of universal design evaluation is outlined.Last but not least, questions and issues regarding thefuture of building performance evaluation are raised.

POST-OCCUPANCY EVALUATION: AN OVERVIEW

A definition of post-occupancy evaluation wasoffered by Preiser et al. (1988): post-occupancy evalu-ation (POE) is the process of evaluating buildings in asystematic and rigorous manner after they have beenbuilt and occupied for some time. The history of POEwas also described in that publication and was summa-rized by Preiser (1999), starting with one-off case studyevaluations in the late 1960s and progressing tosystemwide and cross-sectional evaluation efforts inthe 1970s and 1980s. While these evaluations focusedprimarily on the performance of buildings, the lateststep in the evolution of POE toward building perfor-mance evaluation (BPE) and universal design evalua-tion (UDE) is one that emphasizes a holistic, process-oriented approach to evaluation. This means that notonly facilities, but also the forces that shape them(political, economic, social, etc.), are taken intoaccount. An example of such process-oriented evalua-

tions was the development of the Activation ProcessModel and Guide for hospitals of the Veterans Admin-istration (Preiser, 1997). In the future, one can expectmore process-oriented evaluations to occur, especiallyin large government and private sector organizations,which operate in entire countries or globally, respec-tively.

Many actors participate in the use of buildings,including investors, owners, operators, maintenancestaff, and perhaps most important of all, the end users(i.e., actual persons occupying the building). The focusof this chapter is on occupants and their needs as theyare affected by building performance and on occupantevaluations of buildings. The term evaluation containsthe world “value”; thus, occupant evaluations muststate explicitly whose values are referred to in a givencase. An evaluation must also state whose values areused as the context within which performance will betested. A meaningful evaluation focuses on the valuesbehind the goals and objectives of those who wish theirbuildings to be evaluated, in addition to those who carryout the evaluation.

There are differences between the quantitative andqualitative aspects of building performance and therespective performance measures. Many aspects ofbuilding performance are in fact quantifiable, such aslighting, acoustics, temperature and humidity, durabilityof materials, amount and distribution of space, and soon. Qualitative aspects of building performance per-tain to the ambiance of a space (i.e., the appeal to thesensory modes of touching, hearing, smelling, andkinesthetic and visual perception, including color). Fur-thermore, the evaluation of qualitative aspects of build-ing performance, such as aesthetic beauty or visual


compatibility with a building’s surroundings, is some-what more difficult and subjective and less reliable. Inother cases, the expert evaluator will pass judgment.Examples are the expert ratings of scenic and architec-tural beauty awarded chateaux along the Loire River inFrance, as listed in travel guides. The higher the appar-ent architectural quality and interest of a building, themore stars it will receive. Recent advances in theassessment methodology for visual aesthetic quality ofscenic attractiveness are encouraging. It is hoped thatsomeday it will be possible to treat even this elusivedomain in a more objective and quantifiable manner(Nasar, 1988).

POE is not the end phase of a building project;rather, it is an integral part of the entire building deliv-ery process. It is also part of a process in which a POEexpert draws on available knowledge, techniques, andinstruments in order to predict a building’s likely per-formance over a period of time.

At the most fundamental level, the purpose of abuilding is to provide shelter for activities that couldnot be carried out as effectively, or carried out at all, inthe natural environment. A building’s performance isits ability to accomplish this. POE is the process of theactual evaluation of a building’s performance once inuse by human occupants.

A POE necessarily takes into account the owners’,operators’, and occupants’ needs, perceptions, andexpectations. From this perspective, a building’s per-formance indicates how well it works to satisfy theclient organization’s goals and objectives, as well asthe needs of the individuals in that organization. A POEcan answer, among others, these questions:

• Does the facility support or inhibit the ability ofthe institution to carry out its mission?

• Are the materials selected safe (at least from ashort-term perspective) and appropriate to the useof the building?

• In the case of a new facility, does the buildingachieve the intent of the program that guided itsdesign?

TYPES OF EVALUATION FORBUILDING PROJECTS

Several types of evaluation are made during theplanning, programming, design, construction, andoccupancy phases of a building project. They are oftentechnical evaluations related to questions about the

materials, engineering, or construction of a facility.Examples of these evaluations include structural tests,reviews of load-bearing elements, soil testing, andmechanical systems performance checks, as well aspost-construction evaluation (physical inspection) priorto building occupancy.

Technical tests usually evaluate some physical sys-tem against relevant engineering or performance crite-ria. Although technical tests indirectly address suchcriteria by providing a better and safer building, theydo not evaluate it from the point of view of occupantneeds and goals or performance and functionality asthey relate to occupancy. The client may have a tech-nologically superior building, but it may provide a dys-functional environment for people.

Other types of evaluations are conducted thataddress issues related to operation and management ofa facility. Examples are energy audits, maintenance andoperation reviews, security inspections, and programsthat have been developed by professional facility man-agers. Although they are not POEs, these evaluationsare relevant to questions similar to those describedabove.

The process of POE differs from these and technicalevaluations in several ways:

• A POE addresses questions related to the needs,activities, and goals of the people and organiza-tion using a facility, including maintenance,building operations, and design-related questions.Other tests assess the building and its operation,regardless of its occupants.

• The performance criteria established for POEs arebased on the stated design intent and criteria con-tained in or inferred from a functional program.POE evaluation criteria may include, but are notsolely based on, technical performance specifica-tions.

• Measures used in POEs include indices related toorganizational and occupant performance, such asworker satisfaction and productivity, as well asmeasures of building performance referred toabove (e.g., acoustic and lighting levels, adequacyof space and spatial relationships).

• POEs are usually “softer” than most technicalevaluations. POEs often involve assessing psy-chological needs, attitudes, organizational goalsand changes, and human perceptions.

• POEs measure both successes and failures inher-ent in building performance.

THE EVOLUTION OF POST-OCCUPANCY EVALUATION 11

PURPOSES OF POEs

A POE can serve several purposes, depending on aclient organization’s goals and objectives. POE canprovide the necessary data for the following:

• To measure the functionality and appropriatenessof design and to establish conformance with per-formance requirements as stated in the functionalprogram. A facility represents policies, actions,and expenditures that call for evaluation. WhenPOE is used to evaluate design, the evaluationmust be based on explicit and comprehensive per-formance requirements contained in the func-tional program statement referred to above.

• To fine-tune a facility. Some facilities incorpo-rate the concept of “adaptability,” such as officebuildings, where changes are frequently neces-sary. In that case, routinely recurring evaluationscontribute to an ongoing process of adapting thefacility to changing organizational needs.

• To adjust programs for repetitive facilities. Someorganizations build what is essentially the identi-cal facility on a recurring basis. POE identifiesevolutionary improvements in programming anddesign criteria, and it also tests the validity ofunderlying premises that justify a repetitivedesign solution.

• To research effects of buildings on their occupants.Architects, designers, environment-behaviorresearchers, and facility managers can benefitfrom a better understanding of building-occupantinteractions. This requires more rigorous scien-tific methods than design practitioners are nor-mally able to use. POE research in this caseinvolves thorough and precise measures and moresophisticated levels of data analysis, includingfactor analysis and cross-sectional studies forgreater generalizability of findings.

• To test the application of new concepts. Innova-tion involves risk. Tried-and-true concepts andideas can lead to good practice, and new ideas arenecessary to make advances. POE can help deter-mine how well a new concept works once applied.

• To justify actions and expenditures. Organiza-tions have greater demands for accountability,and POE helps generate the information toaccomplish this objective.

TYPES OF POEs

Depicted in Figure 2-1 is an evolving POE processmodel showing three levels of effort that can be part ofa typical POE, as well as the three phases and ninesteps that are involved in the process of conductingPOEs:

• Indicative POEs give an indication of majorstrengths and weaknesses of a particular building’sperformance. They usually consist of selectedinterviews with knowledgeable informants, aswell as a subsequent walk-through of the facility.The typical outcome is awareness of issues inbuilding performance.

• Investigative POEs go into more depth. Objec-tive evaluation criteria either are explicitly statedin the functional program of a facility or have tobe compiled from guidelines, performance stan-dards, and published literature on a given build-ing type. The outcome is a thorough understand-ing of the causes and effects of issues in buildingperformance.

• Diagnostic POEs correlate physical environmen-tal measures with subjective occupant responsemeasures. Case study examples of POEs at thesethree levels of effort can be found in Preiser et al.(1988). The outcome is usually the creation ofnew knowledge about aspects of building perfor-mance.

The three phases of the post-occupancy evaluationprocess model are (1) planning, (2) conducting, and(3) applying. The planning phase is intended to preparethe POE project, and it has three steps: (1) reconnais-sance and feasibility, (2) resource planning, and(3) research planning. In this phase, the parameters forthe POE project are established; the schedule, costs,and manpower needs are determined; and plans for datacollection procedures, times, and amounts are laid out.

Phase 2—conducting—consists of (4) initiating theon-site data collection process, (5) monitoring andmanaging data collection procedures, and (6) analyz-ing data. This phase deals with field data collection andmethods of ensuring that preestablished sampling pro-cedures and data are actually collected in a manner thatis commensurate with the POE goals.

Furthermore, data are analyzed in preparation for thefinal phase—applying. This phase contains steps(7) reporting findings, (8) recommending actions, and


FIGURE 2-1 Post-occupancy evaluation: evolving performance criteria.

finally, (9) reviewing outcomes. Obviously, this is themost critical phase from a client perspective, becausesolutions to identified problems are outlined and rec-ommendations are made for actions to be taken. Fur-thermore, monitoring the outcome of recommendedactions is a significant step, since the benefits and valueof POEs are established in this final step of the apply-ing phase.

Critical in Figure 2-1 is the arrow that points to“feedforward” into the next building cycle. Clearly, oneof the best applications of POE is its use as input intothe pre-design phases of the building delivery cycle(i.e., needs analysis or strategic planning and facilityprogramming).

BENEFITS, USES, AND COSTS OF POEs

Each of the above types of POEs can result in severalbenefits and uses. Recommendations can be broughtback to the client, and remodeling can be done to cor-rect problems. Lessons learned can influence designcriteria for future buildings, as well as provide infor-mation to the building industry about buildings in use.

This is especially relevant to the public sector, whichdesigns buildings for its own use on a repetitive basis.

The many uses and benefits—short, medium, andlong term—that result from conducting POEs are listedbelow. They refer to immediate action, the three- tofive-year intermediate time frame, which is necessaryfor the development of new construction projects, andthe long-term time frame ranging from 10 to 25 years,which is necessary for strategic planning, budgeting,and master planning of facilities. These benefits pro-vide the motivation and rationale for committing toPOE as a concept and for developing POE programs.

Short-term benefits include the following:

• identification of and solutions to problems infacilities,

• proactive facility management responsive tobuilding user values,

• improved space utilization and feedback on build-ing performance,

• improved attitude of building occupants throughactive involvement in the evaluation process,

• understanding of the performance implications of


changes dictated by budget cuts, and• better-informed design decision-making and

understanding of the consequences of design.

Medium-term benefits include the following:

• built-in capacity for facility adaptation to organi-zational change and growth over time, includingrecycling of facilities into new uses,

• significant cost savings in the building processand throughout the life cycle of a building, and

• accountability for building performance by designprofessionals and owners.

Long-term benefits include the following:

• long-term improvements in building perfor-mance,

• improvement of design databases, standards,criteria, and guidance literature, and

• improved measurement of building performancethrough quantification.

The most important benefit of a POE is its positiveinfluence upon the delivery of humane and appropriateenvironments for people through improvements in theprogramming and planning of buildings. POE is a formof product research that helps designers develop abetter design in order to support changing requirementsof individuals and organizations alike.

POE provides the means to monitor and maintain agood fit between facilities and organizations, and thepeople and activities that they support. POE can alsobe used as an integral part of a proactive facilities man-agement program.

Based on the author’s experience in conductingPOEs at different levels of effort (indicative, investiga-tive, and diagnostic) and involving different levels ofsophistication and manpower, the estimated cost ofthese POEs ranges from 50 cents a square foot forindicative-type POEs to anywhere from $2.50 upwardat the diagnostic level. Some diagnostic-type POEshave cost hundreds of thousands of dollars; such asthose commissioned by the U.S. Postal Service (Farbsteinet al., 1989). On the other hand, indicative POEs, ifcarried out by experienced POE consultants, can costas little as a few thousand dollars per facility and canbe concluded within a matter of a few days, involvingonly a few hours of walk-through activity on site.

The range of charges for investigative-type POEs,

according to the author’s experience, is between$15,000 and $20,000 and covers just about that manysquare feet (Preiser and Stroppel, 1996; Preiser, 1998),amounting to approximately $1.00 per square footevaluated.

The three-day POE workshop format developed bythe author typically costs around $5,000, plus expensesfor travel and accommodation (Preiser, 1996), and ithas proven to be a valuable training and fact-findingapproach for clients’ staff and facility personnel.

AN INTEGRATIVE FRAMEWORK FOR BUILDINGPERFORMANCE EVALUATIONS

In 1997, the POE process model was developed intoan integrative framework for building performanceevaluation (Preiser and Schramm, 1997), involving thesix major phases of the building delivery and life cycles(i.e., planning, programming, design, construction,occupancy, and recycling of facilities). In the follow-ing material, the integrative framework for buildingperformance evaluation is outlined. The time dimen-sion was the major added feature, plus internal reviewor troubleshooting and testing cycles in each of the sixphases.

The integrative framework shown in Figure 2-2attempts to respect the complex nature of performanceevaluation in the building delivery cycle, as well as thelife cycle of buildings. This framework defines thebuilding delivery cycle from an architect’s perspective,showing its cyclic evolution and refinement toward amoving target of achieving better building performanceoverall and better quality as perceived by the buildingoccupants.

At the center of the model is actual building perfor-mance, both measured quantitatively and experiencedqualitatively. It represents the outcome of the buildingdelivery cycle, as well as building performance duringits life cycle. It also shows the six subphases referred toabove: planning, programming, design, construction,occupancy, and recycling. Each of these phases hasinternal reviews and feedback loops. Furthermore, eachphase is connected with its respective state-of-the-artknowledge contained in building type-specific data-bases, as well as global knowledge and the literature ingeneral. The phases and feedback loops of the frame-work can be characterized as follows:

• Phase 1—Planning: The beginning of the build-ing delivery cycle is the strategic plan which


FIGURE 2-2 Building performance evaluation: integrative framework for building delivery and life cycle.

establishes medium- and long-term needs of anorganization through market or needs analysis,which in turn is based on mission and goals, aswell as facility audits. Audits match needed items,including space, with existing resources in orderto establish actual demand.

• Loop 1—Effectiveness Review: Outcomes of stra-tegic planning are reviewed in relation to big-issue categories, such as corporate symbolism andimage, visibility in the context surrounding thesite, innovative technology, flexibility and adap-tive re-use, initial capital cost, operating andmaintenance cost, and costs of replacement andrecycling at the end of the useful life of a building.

• Phase 2—Programming: Once effectivenessreview, cost estimating, and budgeting haveoccurred, a project has become a reality and pro-gramming can begin.

• Loop 2—Program Review: The outcome of thisphase is marked by a comprehensive documenta-tion of the program review involving the client,the programmer, and representatives of the actualoccupant groups.

• Phase 3—Design: This phase contains the stepsof schematic design, design development, andworking drawings or construction documents.

• Loop 3—Design Review: The design phase hasevaluative loops in the form of design review ortroubleshooting involving the architect, the pro-grammer, and representatives of the client orga-nization. The development of knowledge-basedand computer-aided design (CAD) techniquesmakes it possible to apply evaluations during theearliest design phases. This allows designers toconsider the effects of design decisions fromvarious perspectives, while it is not too late tomake modifications in the design.

• Phase 4—Construction: In this phase, construc-tion managers and architects share in construc-tion administration and quality control to ensurecontractual compliance.

• Loop 4—Post-Construction Evaluation: The endof the construction phase is marked by post-construction evaluation, an inspection that resultsin “punch lists,” that is, items that need to be com-pleted prior to commissioning and acceptance ofthe building by the client.

• Phase 5—Occupancy: During this phase, move-in and start-up of the facility occur, as well asfine-tuning by adjusting the facility and its occu-pants to achieve optimal functioning.

• Loop 5—POE: Building performance evaluation


during this phase occurs in the form of POEs car-ried out six to twelve months after occupancy,thereby providing feedback on what works in thefacility and what does not. POEs will assist intesting hypotheses made in prototype programsand designs for new building types, for which noprecedents exist. Alternatively, they can be usedto identify issues and problems in the perfor-mance of occupied buildings and further suggestways to solve these. Furthermore, POEs areideally carried out in regular intervals, that is, intwo- to five-year cycles, especially in organiza-tions with recurring building programs.

• Phase 6—Recycling: On the one hand, recyclingof buildings to similar or different uses hasbecome quite common. Lofts have been con-verted to artist studios and apartments; railwaystations have been transformed into museums ofvarious kinds; office buildings have been turnedinto hotels; and factory space has been remodeledinto offices or educational facilities. On the otherhand, this phase might constitute the end of theuseful life of a building when the building isdecommissioned and removed from the site. Incases where construction and demolition wastereduction practices are in place, building materialswith the potential for re-use will be sorted andrecycled into new products. At this point, hazard-ous materials, such as chemicals and radioactivewaste, are removed in order to reconstitute thesite for new purposes.

UNIVERSAL DESIGN EVALUATION

The concept, framework, and evolution of universaldesign evaluation are based on consumer feedback-driven, preexisting, evolutionary evaluation processmodels developed by the author (i.e., POE and BPE).The intent of UDE is to evaluate the impact on the userof universally designed environments. Working withMace’s definition of universal design, “an approach tocreating environments and products that are usable byall people to the greatest extent possible” (Mace, 1991,in Preiser, 1991), protocols are needed to evaluate theoutcomes of this approach. Possible strategies forevaluation in the global context are presented, alongwith examples of case study evaluations that arepresently being carried out. Initiatives to introduce uni-versal design evaluation techniques in education andtraining programs are outlined. Exposure of students in

the design disciplines to philosophical, conceptual,methodological, and practical considerations of univer-sal design is advocated as the new paradigm for “designof the future.”

UNIVERSAL DESIGN PERFORMANCE

The goal of universal design is to achieve universaldesign performance of designs ranging from productsand occupied buildings to transportation infrastructureand information technology that are perceived to sup-port or impede individual, communal, or organizationalgoals and activities. Since this chapter was commis-sioned by the Federal Facilities Council, the remainderof the discussion will focus on buildings and the builtenvironment as far as universal design is concerned.

A philosophical base and a set of objectives are theseven principles of Universal Design (Center for Uni-versal Design, 1997).

• They define the degree of fit between individualsor groups and their environment, both natural andbuilt.

• They refer to the attributes of products or envi-ronments that are perceived to support or impedehuman activity.

• They imply the objective of minimizing adverseeffects of products, environments, and their users,such as discomfort, stress, distraction, ineffi-ciency, and sickness, as well as injury and deaththrough accidents, radiation, toxic substances,and so forth.

• They constitute not an absolute, but a relative,concept, subject to different interpretations in dif-ferent cultures and economies, as well as temporaland social contexts. Thus, they may be perceiveddifferently over time by those who interact withthe same facility or building, such as occupants,management, maintenance personnel, and visitors.

The nature of basic feedback systems was discussedby von Foerster (1985): The evaluator makes compari-sons between the outcomes (O) which are actuallysensed or experienced, and the expressed goals (G) andexpected performance criteria (C), which are usuallydocumented in the functional program and madeexplicit through performance specifications. VonFoerster observed that “even the most elementarymodels of the signal flow in cybernetic systems requirea (motor) interpretation of a (sensory) signal” and, fur-


ther, “the intellectual revolution brought about bycybernetics was simply to add to a ‘machine,’ whichwas essentially a motoric power system or a sensor thatcan ‘see’ what the machine or organism is doing, and,if necessary, initiate corrections of its actions when go-ing astray.” The evolutionary feedback process inbuilding delivery in the future is shown in Figure 2-3.The motor driving such a system is the programmer,designer, or evaluator who is charged with the responsi-bility of ensuring that buildings meet state-of-the-artperformance criteria.

The environmental design and building delivery pro-cess is goal oriented. It can be represented by a basicsystem model with the ultimate goal of achieving uni-versal design performance criteria:

1. The universal design performance frameworkconceptually links the overall client goals (G),namely those of achieving environmental quality,with the elements in the system that are describedin the following items.

2. Performance evaluation criteria (C) are derivedfrom the client’s goals (G), standards, and state-of-the-art criteria for a building type. Universaldesign performance is tested or evaluated againstthese criteria by comparing them with the actualperformance (P) (see item 5 below).

3. The evaluator (E) moves the system and refers to

such activities as planning, programming, design-ing, constructing, activating, occupying, andevaluating an environment or building.

4. The outcome (O) represents the objective, physi-cally measurable characteristics of the environ-ment or building under evaluation (e.g., itsphysical dimensions, lighting levels, and thermalperformance).

5. The actual performance (P) refers to the perfor-mance as observed, measured, and perceived bythose occupying or assessing an environment,including the subjective responses of occupantsand objective measures of the environment.

Any number of subgoals (Gs) for achieving envi-ronmental quality can be related to the basic system(Preiser, 1991) through modified evaluators (Es), out-comes (Os), and performance (Ps). Thereby, the out-come becomes the subgoal (Gs) of the subsystem withrespective criteria (Cs), evaluators (Es), and perfor-mance of the subsystem (Ps). The total outcome of thecombined basic and subsystems is then perceived (P)and assessed (C) as in the basic system (in Figure 2-4).

PERFORMANCE LEVELS

Subgoals of building performance may be structuredinto three performance levels pertaining to user needs:

EVALUATOR (E) OBJECT OFEVALUATION

OUTCOME: BUILTENVIRONMENT/PRODUCT

(O)

COMPARISON

GOALS(G)

PERFORMANCECRITERIA

(C)

PERFORMANCEMEASURES

(P)

G Goals

CPerformance

Criteria

OOutcome:

Built Environment/ Product

PPerformance

Measures

EEvaluator

FIGURE 2-3 Performance concept/evaluation system.


Per

form

ance

Crit

eria

Pro

gram

mer

/D

esig

ner

Clie

ntS

ub-G

oals

Clie

ntG

oals

Per

form

ance

Crit

eria

Sub

-Goa

ls

Pro

gram

mer

/D

esig

ner

Sub

-Goa

ls

Sys

tem

Per

form

ance

Mea

sure

sS

ub-G

oals

Per

form

ance

Mea

sure

s

Per

form

ance

Crit

eria

S-S

1

Bui

ltE

nviro

nmen

tS

-S 1

Pro

gram

mer

/D

esig

ner

S-S

1

Per

form

ance

Mea

sure

sS

-S 1

Sub

-Sys

tem

1

Bui

ltE

nviro

nmen

t

Per

form

ance

Crit

eria

S-S

2

Bui

ltE

nviro

nmen

tS

-S 1

Pro

gram

mer

/D

esig

ner

S-S

2

Per

form

ance

Mea

sure

sS

-S 2

Sub

-Sys

tem

2

Bui

ltE

nviro

nmen

t

FIG

UR

E 2

-4F

eedb

ack

syst

em w

ith

sub-

syst

ems.


FIGURE 2-5 Evolving performance criteria.

the health-safety-security level, the function and effi-ciency level, and the psychological comfort and satis-faction level. With reference to these levels, a subgoalmight include safety; adequate space and spatial rela-tionships of functionally related areas; privacy, sensorystimulation, or aesthetic appeal. For a number ofsubgoals, performance levels interact and may alsoconflict with each other, requiring resolution.

Framework elements include products-buildings-settings, building occupants and their needs. The physi-cal environment is dealt with on a setting-by-settingbasis. Framework elements are considered in group-ings from smaller to larger scales or numbers or fromlower to higher levels of abstraction, respectively.

For each setting and occupant group, respective per-formance levels of pertinent sensory environments andquality performance criteria are required (e.g., for theacoustic, luminous, gustatory, olfactory, visual, tactile,thermal, and gravitational environments). Also relevantis the effect of radiation on the health and well-being ofpeople, from both short- and long-term perspectives.

As indicated above, occupant needs versus the builtenvironment or products are construed as performancelevels. Grossly analogous to the human needs hierarchy(Maslow, 1948) of self-actualization, love, esteem,safety, and physiological needs, a three-level break-down of performance levels reflects occupant needs in

the physical environment. This breakdown also paral-lels three basic levels of performance requirements forbuildings (i.e., firmness, commodity, delight), whichthe Roman architect Vitruvius (1960) had pronounced.

These historic constructs, which order occupantneeds, were transformed and synthesized into the“habitability framework” (Preiser, 1983) by devisingthree levels of priority depicted in Figure 2-5:

1. health, safety, and security performance;2. functional, efficiency, and work flow perfor-

mance; and3. psychological, social, cultural, and aesthetic per-

formance.

These three categories parallel the levels of stan-dards and guidance designers should or can avail them-selves of. Level 1 pertains to building codes and lifesafety standards projects must comply with. Level 2refers to the state-of-the-art knowledge about products,building types, and so forth, exemplified by agency-specific design guides or reference works such as Time-Saver Standards: Architectural Design Data (Watsonet al., 1997). Level 3 pertains to research-based designguidelines, which are less codified but nevertheless ofimportance for building designers and occupants alike.

The relationships and correspondences between the


habitability framework and the principles of universaldesign devised by the Center for Universal Design(1997) are shown in Figure 2-6.

In summary, the framework presented here system-atically relates buildings and settings to building occu-pants and their respective needs vis à vis the product orthe environment. It represents a conceptual, process-oriented approach that accommodates relational con-cepts to applications in any type of building or envi-ronment. This framework can be transformed to permitstepwise handling of information concerning person-environment relationships (e.g., in the programmingspecification, design, and hardware selection foracoustic privacy).

TOWARD UNIVERSAL DESIGN EVALUATION

The book Building Evaluation Techniques (Baird etal., 1996) showcased a variety of building evaluationtechniques, many of which would lend themselves toadaptation for purposes of UDE. In that same volume,this author (Preiser, 1996) presented a chapter on athree-day POE training workshop and prototype test-

ing module, which involved both the facility plannersand designers and the building occupants (after oneyear of occupancy), a formula that has proven to bevery effective in generating useful performance feed-back data. A proposed UDE process model is shown inFigure 2-7.

Major benefits and uses are well known and include,when applied to UDE, the following:

• Identify problems and develop universal designsolutions.

• Learn about the impact of practice on universaldesign and on building occupants in general.

• Develop guidelines for enhanced universal designconcepts and features in buildings, urban infra-structure, and systems.

• Create greater awareness in the public of suc-cesses and failures in universal design.

It is critical to formalize and document, in the formof qualitative criteria and quantitative guidelines andstandards, the expected performance of facilities interms of universal design.

FIGURE 2-6 Universal design principles versus performance criteria.


FIGURE 2-7 Universal design evaluation: process model with evolving performance criteria.

POSSIBLE STRATEGIES FOR UNIVERSALDESIGN EVALUATION

In the above-referenced models, it is customary toinclude Americans with Disabilities Act (ADA) stan-dards for accessible design as part of a routine evalua-tion of facilities. The ADA standards provide infor-mation on compliance with prescriptive technicalstandards, but say nothing about performance—howthe building or setting actually works for a range ofusers. The principles of universal design (Center forUniversal Design, 1997) constitute an idealistic, occu-pant need-oriented set of performance criteria andguidelines that need to be operationalized. There is alsothe need to identify and consider data-gatheringmethods that include interviews, surveys, direct obser-vation, photography, and the in-depth case studyapproach, among others.

Other authors address assessment tools for universaldesign at the building (Corry, 2001) and urban designscales (Guimaraes, 2001; Manley, 2001). In addition,the International Building Performance Evaluationproject (Preiser, 2001) and consortium created by theauthor has attempted to develop a universal data col-lection tool kit that can be applied to any context andculture, while respecting cultural differences.

The author proposes to advance the state of the artthrough a collection of case study examples of differ-ent building types, with a focus on universal design,including living and working environments, publicplaces, transportation systems, recreational and touristsites, and so forth. These case studies will be structuredin a standardized way, including videotaped walk-throughs of different facility types and with varioususer types. The universal design critiques would focusat the three levels of performance referred to above(Preiser, 1983), i.e., (1) health, safety, and security;(2) function, efficiency, and work flow; and, (3) psy-chological, social, cultural, and aesthetic performance.Other POE examples are currently under developmentthrough the Rehabilitation Engineering and ResearchCenter at the State University of New York at Buffalo.One study focuses on wheelchair users, another, onexisting buildings throughout the United States. ItsWeb site explains that research in more detail(<www.ap.buffalo.edu/>)

Furthermore, methodologically appropriate ways ofgathering data from populations with different levelsof literacy and education (Preiser and Schramm, 2001)are expected to be devised. It is hypothesized thatthrough these methodologies, culturally and contextu-


ally relevant universal design criteria will be developedover time. This argument is eloquently presented byBalaram (2001) when discussing universal design inthe context of an industrializing nation such as India.

The role of the user as “user/expert” (Ostroff, 1997)should also be analyzed carefully. The process of userinvolvement is often cited as central to successfuluniversal design but has not been systematically evalu-ated. Ringaert discusses the key involvement of theuser, as noted above.

EDUCATION AND TRAINING IN UNIVERSALDESIGN EVALUATION TECHNIQUES

Welch (1995) presented strategies for teaching uni-versal design developed in a national pilot projectinvolving 21 design programs throughout the UnitedStates. The initial learning from that project can be usedin curricula in all schools of architecture, industrialdesign, interior design, landscape architecture, andurban design, when they adopt a new approach to em-bracing universal design as a paradigm for design inthe future. In that way, students will be familiarizedwith the values, concept, and philosophy of universaldesign at an early stage, and through field exercisesand case study evaluations, they will be exposed to real-life situations. As noted in Welch, it is important tohave multiple learning experiences. Later on in the cur-riculum, these first exposures to universal designshould be reinforced through in-depth treatment of thesubject matter by integrating universal design into thestudio courses, as well as evaluation and programmingprojects.

A number of authors, including Jones (2001),Pedersen (2001), and Welch and Jones (2001), offercurrent experiences and future directions in universaldesign education and training.

CONCLUSIONS

For universal design to become viable and trulyintegrated into the building delivery cycle of main-stream architecture and the construction industry, it willbe critical to have all future students in these fieldsfamiliarized with universal design, on one hand, and todemonstrate to practicing professionals the viability ofthe concept through a range of POE-based UDEs,including exemplary case study examples, on the other.

The “performance concept” and “performancecriteria” made explicit and scrutinized through post-

occupancy evaluations have now become an acceptedpart of good design by moving from primarily subjec-tive, experience-based evaluations to more objectiveevaluations based on explicitly stated performancerequirements in buildings.

Critical in the notion of performance criteria is thefocus on the quality of the built environment asperceived by its occupants. In other words, buildingperformance is seen to be critical beyond aspects ofenergy conservation, life-cycle costing, and the func-tionality of buildings: it focuses on users’ perceptionsof buildings.

For data-gathering techniques for POE-based UDEsto be valid and standardized, the results need to becomereplicable.

Such evaluations have become more cost-effectivedue to the fact that shortcut methods have been devisedthat allow the researcher or evaluator to obtain validand useful information in a much shorter time framethan was previously possible. Thus, the cost of staffingevaluation efforts, plus other expenses have been con-siderably reduced, making POEs affordable, especiallyat the “indicative” level described above.

ABOUT THE AUTHOR

Wolfgang Preiser is a professor of architecture at theUniversity of Cincinnati. He has more than 30 years ofexperience in teaching, research, and consulting, withspecial emphasis on evaluation and programming ofenvironments, health care facilities, public housing,universal design, and design research in general.Dr. Preiser has had visiting lectureships at more than30 universities in the United States and more than35 universities overseas. As an international buildingconsultant, he was cofounder of Architectural ResearchConsultants and the Planning Research Institute, Inc.,both in Albuquerque, New Mexico. He has written andedited numerous articles and books, including Post-Occupancy Evaluation and Design Intervention:Toward a More Humane Architecture. Dr. Preiser is agraduate fellow at the University of Cincinnati. Hereceived the Progressive Architecture Award and Cita-tion for Applied Research, and the EnvironmentalDesign Research Association (EDRA) career award.In addition, he was a Fulbright fellow and held twoprofessional fellowships from the National Endowmentof the Arts. He is a member of the editorial board ofArchitectural Science Review; associate editor of theJournal of Environment and Behavior; and a member


and former vice-chair and secretary of EDRA. He iscofounder of the Society for Human Ecology (1978).In the mid-1980s, he chaired the National ResearchCouncil Committee on Programming Practices in theBuilding Process and the Committee on Post-Occupancy Evaluation Practices in the Building Pro-cess. Dr. Preiser holds a bachelor’s degree in architec-ture from the Technical University, Vienna, Austria; amaster of science in architecture from Virginia Poly-technic Institute and State University; a master ofarchitecture from the Technical University, Karlsruhe,Germany; and, a Ph.D. in man-environment relationsfrom Pennsylvania State University.

REFERENCESBaird, G., et al. (Eds.) (1996). Building Evaluation Techniques. London:

McGraw-Hill.Balaram, S. (2001). Universal design and the majority world. In: Preiser,

W.F.E., and Ostroff, E. (Eds.) Universal Design Handbook. New York:McGraw-Hill.

Center for Universal Design (1997). The Principles of Universal Design(Version 2.0). Raleigh, N.C.: North Carolina State University.

Corry, S. (2001). Post-occupancy evaluation and universal design. In:Preiser, W.F.E., and Ostroff, E. (Eds.) Universal Design Handbook.New York: McGraw-Hill.

Farbstein, J., et al. (1989). Post-occupancy evaluation and organizationaldevelopment: The experience of the United States Post Office. In:Preiser, W.F.E. (Ed.) Building Evaluation. New York: Plenum.

Guimaraes, M.P. (2001). Universal design evaluation in Brazil: Developingrating scales. In: Preiser, W.F.E., and Ostroff, E. (Eds.) UniversalDesign Handbook. New York: McGraw-Hill.

Jones, L. (2001). Infusing universal design into the interior design curricu-lum. In: Preiser, W.F.E., and Ostroff, E. (Eds.) Universal Design Hand-book. New York: McGraw-Hill.

Mace, R., G. Hardie, and J. Place (1991). Accessible Environments: Towarduniversal design. In: Design Intervention: Toward a More HumaneArchitecture. Preiser, W.F.E. Vischer, J.C. and White. E.T. (Eds.) NewYork: Van Nostrand Reinhold.

Manley, S. (2001). Creating an accessible public realm. In: Preiser, W.F.E.,and Ostroff, E. (Eds.) Universal Design Handbook. New York:McGraw-Hill.

Maslow, H. (1948). A theory of motivation. Psychological Review 50:370-398.

Nasar, J.L. (Ed.) (1988). Environmental Aesthetics: Theory, Methods andApplications. Cambridge, Mass.: MIT Press.

Ostroff, E. (1997). Mining our natural resources: the user as expert. Innova-tion, The Quarterly Journal of the Industrial Designers Society ofAmerica 16(1).

Pedersen, A. (2001). Designing cultural futures at the University of WesternAustralia. In: Preiser, W.F.E., and Ostroff, E. (Eds) Universal DesignHandbook. New York: McGraw-Hill.

Preiser, W.F.E. (1983). The habitability framework: A conceptual approachtoward linking human behavior and physical environment. DesignStudies 4 (No. 2)

Preiser, W.F.E. Rabinowitz, H.Z., and White, E.T. (1988). Post-OccupancyEvaluation. New York: Van Nostrand Reinhold.

Preiser, W.F.E. (1991). Design intervention and the challenge of change.In: Preiser, W.F.E., Vischer, J.C., and White, E.T., (Eds.) Design Inter-vention: Toward a More Humane Architecture. New York: VanNostrand Reinhold.

Preiser, W.F.E. (1996). POE Training Workshop and Prototype Testing atthe Kaiser-Permanente Medical Office Building in Mission Viejo, Cali-fornia, USA. In Baird, G., et al. (Eds.) Building Evaluation Techniques.London: McGraw-Hill.

Preiser, W.F.E., and Stroppel, D.R. (1996). Evaluation, reprogramming andre-design of redundant space for Children’s Hospital in Cincinnati. In:Proceedings of the Euro FM/IFMA Conference, Barcelona, Spain, May5-7.

Preiser, W.F.E. (1997). Hospital activation: Towards a process model.Facilities 12/13; 306-315.

Preiser, W.F.E. and Schramm, U. (1997). Building performance evaluation.In: Watson, D., Crosbie, M.J. and Callendar, J.H. (Eds.) Time-SaverStandards: Architectural Design Data. New York: McGraw-Hill.

Preiser, W.F.E. (1998). Health Center Post-Occupancy Evaluation: TowardCommunity-Wide Quality Standards. Sao Paulo, Brazil: Proceedings ofthe NUTAU/USP Conference.

Preiser, W.F.E. (1999). Post-occupancy evaluation: Conceptual basis, ben-efits and uses. In: Stein, J.M., and Spreckelmeyer, K.F. (Eds.) ClassicalReadings in Architecture. New York: McGraw-Hill.

Preiser, W.F.E. (2001). The International Building Performance Evaluation(IBPE) Project: Prospectus. Cincinnati, OH: University of Cincinnati.Unpublished Manuscript.

Preiser, W.F.E. and Schramm, U. (2001). Intelligent office building perfor-mance evaluation in the cross-cultural context: A methodological out-line. Intelligent Building I(1).

Vitruvius (1960). The Ten Books on Architecture (translated by M.H.Morgan) New York: Dover Publications.

von Foerster, H. (1985). Epistemology and Cybernetics: Review andPreview. Milan: Casa della Cultura.

Watson, D., Crosbie, M.J., and Callender, J.H. (Eds.) (1997). Time-SaverStandards: Architectural Design Data. New York: McGraw-Hill (7th

Edition).Welch, P. (Ed.) (1995). Strategies for Teaching Universal Design. Boston,

Mass: Adaptive Environments Center.Welch, P., and Jones, S. (2001). Teaching universal design in the U.S. In:

Preiser, W.F.E., and Ostroff, E. (Eds.) Universal Design Handbook.New York: McGraw-Hill.

23

3

Post-Occupancy Evaluation:A Multifaceted Tool for Building Improvement

Jacqueline Vischer, Ph.D., University of Montreal

WHAT IS POST-OCCUPANCY EVALUATION?

Various definitions of Post-Occupancy Evaluation(POE) have been advanced over the last 20 years sincethe term was coined. Loosely defined, it has come tomean any and all activities that originate out of aninterest in learning how a building performs once it isbuilt, including if and how well it has met expectationsand how satisfied building users are with the environ-ment that has been created. POEs can be initiated asresearch (Marans and Spreckelmeyer, 1981), as casestudies of specific situations (Brill et al., 1985), and tomeet an institutional need for useful feedback on build-ing and building-related activities (Farbstein andKantrowitz, 1989). For some public agencies, such asthe State of Massachusetts and Public Works Canada(now Public Works and Government Services Canada),POE is a mechanism for linking feedback on newlybuilt buildings with pre-design decision-making; thegoal is to make improvements in public buildingdesign, construction, and delivery.

Evidence from POE activities to date indicates thatobjectives such as finding out how buildings work oncethey are built—and whether the assumptions on whichdesign, construction, and cost decisions were based arejustified—are primarily of interest to large institutionalowners and managers of real estate inventory. Tenantorganizations, small owner-occupiers, and privatesector commercial property managers are not typicallyinvestors in POE. Moreover, the number of large insti-tutional owners and managers of real estate who haveactive POE programs is extremely small.

THE PROS AND CONS OF POE

One of the characteristics of POE activities is thediscrepancy that exists between the reasons for doingPOE (pros) and the difficulty of doing them (cons).Reasons for doing POEs are well represented in theliterature. One reason is to develop knowledge aboutthe long-term and even the short-term results of designand construction decisions—on costs, occupant satis-faction, and such building performance aspects asenergy management, for example. Another reason is toaccumulate knowledge so as to inform and improve thepractices of building-related professionals such asdesigners, builders, and facility managers and even toinform the clients and users who are the consumers ofservices and products of those same building-relatedprofessionals. For an institutional owner-manager ofreal estate (government agencies, large quasi-governmentorganizations), POE studies can provide feedback onoccupant satisfaction, on building performance, and onoperating costs and management practices. In sum,POE is a useful tool for improving buildings, increas-ing occupant comfort, and managing costs. So whatmitigates against POE being a more universal activity?The barriers to widespread adoption of POE are cost,defending professional territory, time, and skills. Eachone of these is examined briefly.

Cost

The cost barrier is not caused by the high costs ofdoing POE: building evaluation studies can be asexpensive or as inexpensive as the resources availableto finance them. The cost barrier is intrinsic to the


structure of the real estate industry, namely, who paysfor POE? In commercial real estate circles, POE is notbuilt into the architect’s fee, the construction bid, themove-in budget, or the operating budget of the build-ing. This means that money to finance any POEactivity, however small, must be found on a case-by-case basis.

Professional Territory

Defending professional territory is a barrier becausePOE is, after all, evaluation, and evaluation impliesjudgment. No active building professionals seek tohave their work judged by outsiders as part of a processover which they have no control, even if the goal is abetter understanding of a situation and not a perfor-mance review of a participant. It is necessary for POEto be seen as a useful a posteriori gathering of knowl-edge that is of value to the professionals involved, notas a critique of professional performance.

Time

The question of time is a mysterious one in commer-cial real estate. Every new building project has a rushedand constraining schedule, and every stage is carriedout under unbending time pressures, although thereasons for this are not always clear at the time and inspite of the fact that rushed and fast-tracked projectsoften lead to costly change orders and bad long-termdecisions. Going back for a follow-up look at a build-ing, however, is not bound by the time pressures ofnew projects and, as a result, finds no place in thephases of a conventional building project.

Skills

Finally, what are POE skills and why is the lack ofthem a barrier? In spite of considerable reflection andwriting by academics and researchers, there is no par-ticular technique or tool associated with POE studies.The result is that the term itself has come to be appliedto a wide range of different activities, ranging fromprecise cost-accounting evaluations to technical mea-surements of building performance to comprehensivesurveys of user attitudes. Defining skills so broadlymeans that no one individual is likely to have all thatare needed; it also means that POE does not fall intothe skill set of any one individual or discipline andtherefore tends to fall through the cracks.

In spite of the power of both the pros and cons ofPOE, the activity continues to be legitimized by one-off studies commissioned by large-scale owner-occupiersas well as by companies who, for one reason or another,are seeking to make more strategic real estate decisions.The term continues to have currency among academicresearchers who are motivated to add to the generalknowledge base about how buildings work after occu-pancy and how the environments they offer affectusers. In the current context of new work patterns,changing office technology, and a more strategicapproach to workspace planning, POE studies are findinga potentially valuable role in guiding companies towardmore informed decision-making about office space.

CURRENT STATUS OF POE

POE has evolved from early efforts at environmentalevaluation that focused on the housing needs of dis-advantaged groups and efforts to improve environmen-tal quality in government-subsidized housing. The ideathat better living space could be designed by havingbetter information from users drove environmentalevaluation in Britain, France, Canada, and the UnitedStates during the 1960s and into the 1970s. It was onlyafter the widespread acceptance of this logic—thatfinding out about users’ needs was a legitimate aim ofbuilding research—that other building types becametargets for evaluation, namely, public buildings, includ-ing courthouses, prisons, and hospitals.

The building type most recently identified as a can-didate for POE is office and commercial buildingdesign. Starting with the BOSTI study (Brill et al.,1985) linking features of the office environment toemployee productivity, the corporate preoccupationwith reducing space costs and improving productivityhas caused the private sector to become more activelyinvolved in POE. The challenge is to build POE intothe cycle of corporate real estate decision-making sothat professionals involved in building programming,design, construction, and operation can acquire the rel-evant tools and skills; so that provision for POE is builtinto either the operating or the capital budget; and sothat the results of a POE feed into decision-making in auseful and constructive way.

In the following sections, four types of POE areidentified. Each one is illustrated with at least one casestudy showing how it has been used. Although thesefour categories of POE are not exhaustive, they seemmost useful for this overview. They are

POST-OCCUPANCY EVALUATION: A MULTIFACETED TOOL FOR BUILDING IMPROVEMENT 25

1. building-behavior research, or the accumulationof knowledge;

2. feeding into pre-design programming;3. strategic space planning; and4. capital asset management.

At the end, the “best practices” that can be identifiedfrom this comparative analysis are summarized. Ulti-mately, the process of POE is seen as critical in termsof meeting the challenges identified above. In the lastsection, a functionally viable POE process is outlined.

Building-Behavior Research, or the Accumulationof Knowledge

The notion of POE as a routine activity of the realestate industry has not gained ground in Europe, whereit remains an active area of applied research in mostcountries. At a seminar in Paris in 1992, French policy-makers, public servants, and administrators wereexposed to a rich panoply of North American POEresearch on public buildings in order to demonstratethe value of the approach and the increasing knowl-edge about buildings (Centre Scientifique et techniquedu bâtiment, 1993). In most European circles, the ideaappears to be limited to individual academic researcherswho carry out housing research,1 some office buildingstudies,2 and public building POEs,3 as well as a grow-ing number of hospital POEs in Sweden, Germany, andEngland (Dilani, 2000).

Funded by government agencies and using academi-cally defensible research methods to study largelypublic building use, POE in Europe and Japan seems tobe directed ultimately at building a broader and morereliable base of knowledge of human behavior in rela-tion to the built environment, knowledge that mayeventually come to be recognized as an academic dis-cipline (environmental psychology, interior design?)but is not actively channeled to designers or other pro-fessionals in the real estate industry.

POE was identified as a component of the ProjectDelivery System used by Public Works Canada in theearly 1980s, and was intended as a final stage in theprogramming, design, construction, and occupancy

process of federal projects. A multidisciplinaryapproach to POE was developed and implemented fora short time in different federal office buildings inCanada (Public Works Canada, 1983). Precipitated bya concern with energy consumption in the early 1980s,studies were initiated of the performance of buildingsystems, patterns of energy use in large buildings, andeffects on occupants’ perceptions of comfort. Thesestudies led to methods to devise effective but simplifieddata-gathering methods to provide reliable indicatorsof building quality (Ventre, 1988). The glue that boundthese data-gathering and analysis efforts together wasan analysis of user behavior and the links that could orcould not be made with building operations.

A technique for assessing user comfort was one toolthat emerged out of the Canadian effort and has sincebeen widely implemented in private industry (Dillonand Vischer, 1988). An extensive survey of users wasinitiated in some eight government buildings in Cana-dian cities, and a major data analysis effort aimed tointegrate the feedback from users with data collectedfrom instruments measuring indoor air quality, thermalcomfort, lighting and acoustic conditions, and energyperformance. Analysis of the questionnaire results ledto the conclusion that there are seven major conditionsthat affect users’ perceptions of their comfort in officebuildings; each can be related to measures of perfor-mance of technical building systems, but not in director causal ways (Vischer, 1989).

The identification of what came to be known as theBuilding-In-Use measure of ambient environmentalcomfort led to the development of a standardizedmeasurement tool in the form of a short questionnaire.The questions are formatted as 5-point scales on whichbuilding occupants rate the seven key dimensions ofenvironmental comfort in their workplace. Both thefive-year data-gathering and analysis effort that led upto the building-in-use system and its subsequent exten-sive use in the private sector can be considered a majorPOE initiative that has important implications forbuilding-behavior research and has also generated atool that can be used for other types of POE (Vischer,1996).

Since its development for the Canadian government,the Building-In-Use (BIU) assessment system has beenused all over the world. Two books in English and onein French, along with several articles, have been pub-lished that describe the system and its applications. Acopy of the questionnaire is contained in Appendix C.Subsequent sections of this chapter deal with applica-

1For example, Mirella Bonnes at the University of Rome in Italy.2For example, Peter Jockusch at the University of Kassal,

Germany.3For example, the Building Research Institute in Olso, Norway.


tions of this POE system in a variety of differentcontexts.

This year, a new research initiative in Quebec,Canada, has identified among other objectives the needto update and modernize the BIU assessment system.This objective is combined with another broad-rangingresearch goal, that of carrying out a POE of some 3,000universal work-station installations in the offices ofQuebec’s largest insurance company. Not only is thisPOE targeting ambient environmental conditions in thework environment, as was done in the 1980s, but it willalso examine the psychological impact on individual andgroup work of a highly standardized work environment.

This study aims to make a valuable contribution tothe office POE literature by adding to existing knowl-edge of building systems performance and human com-fort. As well as updating our knowledge of key envi-ronmental conditions in the workplace, this researchwill measure psychological needs such as privacy andterritoriality and the influences of group norms andmembership as well as organizational values on occu-pant perception of the work environment. More purelysocial science methods are being used, such as indi-vidual interviews, focus groups, and a questionnairesurvey to be carried out through individual interviewswith a stratified random sample of the populations ofup to six different buildings. Results will become avail-able in the fall of 2001, and the study will be publishedin 2002.

Linking POE with Pre-Design Programming

One of the most appealing reasons to perform POEis to be able to inform building decision-making in theearly stages of a new project. POE studies target userevaluation of an existing space where users are des-tined to occupy a new space that is being planned. Theirfeedback is needed to ensure that the new design meetsusers’ needs and solves problems in existing buildings.Certain public agencies such as the Division of CapitalPlanning and Operations in Massachusetts have POEas a legitimate and funded stage in all capital projects;the activity is run by the Office of Programming, theoffice responsible for all pre-design planning. The con-cept behind the legislation was to link POE with pre-design programming of public buildings.

In certain projects, the link has been effected andhas paid off. For example, State Police stations are alldesigned along the same principles because they allserve the same functions. The Office of Programming

developed a prototype concept that was built and occu-pied. The post-occupancy evaluation was part of thedesign process for new police stations, and the proto-type was carefully examined in use, with its function-ality, costs, structure, and materials evaluated. Theprototype design was then modified, and the design ofstate police stations was standardized and built alongthe same lines. This saved time and effort on program-ming, design, and construction costs for the state. Asimilar approach has worked for child care centers andstate vehicle repair and maintenance centers, and wasbeing considered for state armories and firing ranges.

Soon after a large sum was approved by the Massa-chusetts legislature for a fast-tracked program of newprison construction, a post-occupancy prison study wascarried out. The results of the new Old Colony statepenitentiary POE were delivered to the Department ofCorrections and ultimately used in programming fourto six fast-tracked corrections projects by the Office ofProgramming.

However, in other projects, POE was not as success-ful. It was not uncommon for the budgeted amount forPOE to be used up by change orders and other require-ments of the construction process. These projects hadno funds left for the POE stage of the process. In othercases, the time barrier alluded to above created a mis-alignment between POE studies and programming anddesign activities. For example, POEs of the correctionalinstitutions built early in the fast-tracked process werenot done because their results would not have beenavailable in time to inform programming and designfor the next project. A POE on a Massachusetts court-house in 1985-1986 was only approximately alignedwith the state’s courthouse construction program thatwas funded from 1984-1989.

POEs are still part of the public building program-ming process, and efforts have recently been made todevelop and implement a standardized POE procedurethat will fit in with the state’s building programs, pro-vide the right information at the right time, and enableproject managers to identify more exactly for eachproject the amount of money needed for POE.

POE was also built into government building deliv-ery processes in New Zealand in the 1980s before theprivatization of the public works department. Perform-ing POEs facilitated pre-design programming and gavethe design and construction team on each project acloser contact and understanding of users in each of itsprojects. This led to a design approach characterizedby the designers as a negotiation, with multiple


exchanges of information and openness to change onboth sides (Joiner and Ellis, 1989). However, thisapproach was threatened by the severe government cut-backs of the 1980s, closely followed by extensiveprivatization that put the public works departments incompetition with private firms for public buildingprojects. Although the close links with users that thepublic works department had developed gave it anedge, the time needed for negotiated design was notcompetitive, even though it could demonstrate thatsavings would be realized later on in the life of thebuilding.

These accounts suggest that in spite of the logicalimperative to link POE results to the front end of thedesign process, efforts to do so have had to struggle tosurvive. This should not be taken to mean that suchefforts are futile; on the contrary, they are valiant andshould be continued, if only to make us question thebasic irrationality of the present building creationprocess.

POE in Strategic Space Planning

There is a clear difference between attempts to feedPOE study results into pre-design decision-making ona routine basis, such as those described above, andusing POE in strategic space planning. This latter useof POE has gained credibility in recent years as corpo-rations are trying increasingly to provide functionallysupportive workspace to their employees and simulta-neously to reduce occupancy costs.

A number of companies in recent years have usedBuilding-In-Use assessment to initiate a process of stra-tegic space planning. This approach indicates a morecomplex situation than that which is characterized byconventional office space planning. It implies that theorganization seeks to improve, innovate, or otherwiseinitiate workspace change to bring space use more inline with strategic business goals. These changes arenot always understood or accepted by employees; thus,some companies have taken a change-managementapproach to new space design. Others have imposedmajor workspace changes in the same way as any otherredesign of the workspace, sometimes with markedlynegative results (Business Week, 1996).

One example of a company that used POE for strate-gic space planning in the early 1990s and has realizedsignificant gains in corporate recruitment and retention,as well as increased sales, is Hypertherm Inc. (Zeisel,in press). This medium-sized manufacturing company

located in New Hampshire is a world leader in thedesign and production of plasma metal-cutting equip-ment. As a result of a need for expansion of both itsmanufacturing plant and its offices, Hypertherm hiredan architect whose drawings and conceptual approachit later rejected on the grounds that a new work envi-ronment should accompany the significant organiza-tional change that was needed to prepare the companyfor global expansion and a better competitive positionas its share of the world market grew.

The workspace design process at Hypertherm includeda POE of the existing space. As well as providing use-ful data to help design the new space, the survey causedemployees to feel both consulted and involved in theprocess of designing new space that would meet busi-ness goals. The strategic space planning comprised anumber of different steps. First, a shared vision of thenew work environment was created through a struc-tured team walk-through of the existing facility. Themanagement team and consultants toured the facilityas a group, discussing the tasks of each work group,pointing out difficulties and advantages with thepresent space, and commenting on each other’s presen-tations. Recorded on cassette tape and transcribed, thetour commentary was presented back to the client,along with a comprehensive set of photographs, todocument existing conditions. This activity involvedall members of the team and gave the consultant a largeamount of information efficiently; consensus began tobuild on what needed to be done to solve the company’sspace problems.

A subsequent series of facilitated work sessionsenabled the management team to generate a set of goalsand objectives not only for the new space but also forthe restructured organization. This stage yielded a setof design guidelines to be applied to the new designand established priorities as to the relative importanceof what the team wanted to achieve. Most importantlyfor the future of the process, it resulted in consensus.

The next step required involving employees in thedesign process in order to ensure widespread accep-tance of the vision. The BIU assessment survey ofemployee perceptions of the physical conditions of theexisting building started the process of employeeinvolvement. Each employee filled out the survey andwas therefore alerted to the imminent new space designprocess and to the importance of his or her role in it.The survey results were published in the companynewsletter and showed the best and the worst aspectsof working in the old space.


FIGURE 3-1 Before and after BIU profile comparison.

2.6

3.3

3

3.3

2.8

3.4

2.9

2.6

3.73.9

2.6

2.9

4.34.5

2.8

3.4

1

2

3

4

5

Occ

upan

t Com

fort

Air Quality ThermalComfort

Spatial Comfort

Privacy Lighting Comfort

Office Noise

Building Noise

Workable

Hypertherm 1996 Hypertherm 1998

A final step in the strategic approach was to inviteemployee representatives to provide feedback ondesign development and to communicate key designdecisions to their colleagues.

The questionnaire survey was distributed a secondtime, about six months after move-in, to compare levelsof user comfort between the old and the new buildings.Not only were occupants pleased with their new space,but the process of buy-in and participation also helpedthem understand from day one how it would work.They accepted it without the discomfort and resistanceoften exhibited in new work environments, becausethey knew exactly what to expect. The employees tookownership of their new space because they had beeninvolved in decision-making throughout.

Figure 3-1 shows employee ratings on the sevendimensions of workspace comfort in the old buildingand after the renovation. Having a short, standardizedquestionnaire to compare employee perceptions beforeand after a workspace change is a significant tool forPOE studies. BIU assessment has been used success-fully in strategic space planning for a range of compa-nies, including Bell Sygma, Reuters, Boston FinancialGroup, and GTE Government Systems.

Capital Asset Management

Using POE as a tool for managing building assets isnot new, but it has not been widely implemented. This

is perhaps due to the diverging focus of the two activi-ties. Asset management tends to rely on data on build-ing operating costs, maintenance and repair needs, realestate value and market conditions, and tenant improve-ments. The standard POE approach provides data onuser perceptions and attitudes through group feedbacksessions, such as focus groups, survey questionnaires,and in-situ observations of user behavior. As a result,feedback from employees in corporations is often con-sidered to be of interest more to human resourcesdepartments than to the real estate team.

However, an approach to POE that combines assess-ment of the physical condition of the building andbuilding systems with user comfort assessment on suchtopics as indoor air quality and ventilation rates, light-ing levels and contrast conditions, building (not occu-pant) noise levels, and indoor temperature (thermalcomfort) could constitute another tool to add to thoseused in conventional asset management. One weaknessin making this potentially fruitful link is the findingthat much of the literature that has been published onspecific building studies has been unable to demon-strate systematic links between the feedback usersprovide through questioning and data derived frominstrument measures of interior building conditions(Vischer, 1993). Moreover, portfolio managers tend tosteer away from POE approaches because there are fewtools available (the BIU assessment questionnairebeing one of the few simple ways of collecting user


feedback on a standardized basis) and because anyquestioning of users requires informing and involvingtenants. A more finely tuned and precise approach toPOE is necessary in order to make this approach valu-able to professional evaluators and asset managers inorganizations.

However, BIU assessment has been used by largeowner-occupier corporations to collect data on userperceptions and link these directly with ambient inte-rior conditions and, therefore, building performanceand building quality. Two examples of companies withextensive real estate holdings who attempted to usePOE systematically as an asset management tool areBell Canada and the World Bank. Both organizationsowned some and leased some of their office space; bothwere committed to providing high-quality office space,in the first case, as part of a continuous improvementphilosophy and in the second case, as part of the corpo-rate mission; and both chose BIU assessment as theirPOE tool for managing assets.

In both cases, BIU surveys were carried out inalmost all of their buildings, in both leased and owner-occupied space. In the case of Bell, this meant surveysof 2,500 people distributed in its headquarters tower inMontreal, in suburban office buildings in Montreal andToronto, and in one building in Quebec City. In thecase of the World Bank, this meant surveying 2,800employees distributed in the eight leased and ownedbuildings it occupied in Washington, D.C., and alsoincluded one building in Paris.

Both corporations collected large amounts of feed-back from occupants using the BIU questionnaire. Inaddition to individual building analysis, the BIU dataon environmental comfort were grouped and analyzedfor overall trends in occupant comfort. The large numberof cases and variety of building settings surveyed enabledbaseline scores to be calculated on the seven comfortdimensions across all buildings. This, in turn, allowedreal estate staff to identify which buildings or parts ofbuildings exceeded the baseline scores and which fellbelow them. Both organizations found that this approachwas cost-effective and not data heavy, did not consumeinappropriate amounts of staff time, and provided asingle-digit indicator of environmental quality. BIUresults from individual buildings could easily be com-pared either to their own baseline, that is, the standardsset by their own building stock, or to the baseline scoresgenerated by the pre-existing BIU database, indicatinga generalized North American standard of quality basedon survey results from some 60 buildings.

Procedures were set in place to allow the baselinescores to be updated as new information was addedthrough additional building surveys. One organization,Intelsat, initiated an electronic form of the question-naire survey in order to be able to update occupant com-fort ratings easily. In the case of the World Bank, aneffort was made to link the BIU database to computer-ized drawings that were used to plan and update officelayouts, so that BIU scores of buildings, floors, or areasof floors that were slated for reconfiguration could beconsulted and indices of quality made available as partof the space planning process—a sort of instant POE.

BEST PRACTICES

The rapid overview of case studies presented aboveoffers the following conclusions. First, it is clear thatPOEs of built environments must continue in order toenhance our knowledge about the effects of physicalspace on people. The challenge of the “building-behavior research” definition of POE is to ensure thatthe knowledge gained from research studies is not onlydisseminated in the academic community but also suc-cessfully transferred to the world of designers, builders,and financiers of real estate.

For public agencies or other organizations thatrepeatedly construct the same building type, linkingPOE with pre-design programming can save moneyand time. Evidence indicates, however, that even whenthe link to pre-design decision-making is recognized,POE is not simple to implement. It is likely to be suc-cessful only if, as Friedman et al. (1979) pointed out intheir seminal first book on building evaluation, astructure-process approach is used. This means design-ing an approach ahead of time, developing and testingthe process beforehand, and ensuring that resourcescontinue to be available.

POE is also a potentially useful tool for asset man-agement, as long as the approach employed to collectfeedback from users can be effectively integrated withthe other more market-oriented data-gathering effortsof asset managers. This may mean simplifying theelaborate social science approach favored by researchersand investing in a test initiative to implement and testan asset management approach to POE in the contextof the real estate industry so as to demonstrate howfeedback from users can both be collected easily andenhance real estate decision-making.

POE also seems to have a natural place in strategicspace planning and could be developed for use by a


wide variety of organizations. The key to this applica-tion is to consider POE a tool for involving buildingusers in planning new workspace. Some organizationshave reservations about allowing their employees tobecome too involved in the emotional and time-consuming planning of their new space. However, tech-niques exist for managed participation that have beensuccessful in a variety of instances in helping to controlthe amount and type of user involvement. Involvingusers in new workspace planning is necessary for anysuccessful change initiative, and POE is one of the toolsavailable to this end.

Finally, in spite of the ground-breaking efforts bysome large organizations to build POE techniques intotheir building management activities, it is curious thatlarge property management firms are rarely known touse POEs for building diagnostic purposes or forimproving services to tenants. Some property manage-ment companies content themselves with short satis-faction questionnaires that tenants complete followinga repair or move. However, in the experience of thiswriter, property management firms, along with largebanks and financial institutions, are the companies leastlikely to perform POE. Techniques of POE need to bedeveloped for use by organizations that would makegood use of occupant feedback if they had a simple,reliable way of getting it. These techniques would helpthem to build environmental evaluation into their plan-ning, budgeting, and maintenance cycles.

It should be noted that some companies fear solicit-ing feedback from building occupants on the groundsthat both seeking and receiving this type of informa-tion may obligate them as building owners and/or man-agers to make a costly change to their services or to thebuilding itself. At least one lawsuit has been heard of,resulting from a perceived lack of follow-up to anoccupant survey that questioned users about their per-ceptions of indoor air quality and lighting (BostonGlobe, 1987).

MANAGING POE INFORMATION

Once POE exists outside the protected frameworkof a case study research project, another set of barrierspresent themselves in the form of dissemination of theinformation yielded by the study. As long as the POE iscarried out as academic research, the sanctioned formsof academic research dissemination are available (pub-lication in journals, conferences, etc.). However, in thepractical world of building design, construction, and

management, most organizations have no establishedsystem for knowing how to process, direct, and act onthe information they receive from a POE. This maycause the information not to go anywhere, and itbecomes a reminder to decision-makers not to repeatthe experience. Having no clear use for the informationmay generate conflict and resentment among those whoare expected to act on it; seeing their feedback ignoredand not put to good use may alienate building occupants.

Many organizations that initiate POE are unclear asto why they want the information, what informationthey want, to whom it should go, and how they areexpected to follow up on it. Several organizationsfamiliar to the author have explicitly required that theresults—whether positive or not—of a POE survey notbe disseminated.

Among the range of possible reasons for a lack ofplanning for the dissemination of POE information arethe following:

• “usefulness” of user surveys,• complexity of the design process,• negativity of the comments received, and• complexity of managing information.

Each of these is discussed below.

Usefulness of User Surveys

Questioning people about how much they like or dis-like as space that they occupy inevitably obliges theresearcher to confront the “so-what?” question: Sowhat if some users like a feature and others do not? Thenotion of liking something is so subjective and con-strained by circumstances that it is difficult to extractgeneric information or to generalize from users’responses. However the notion of user satisfaction is atthe base of almost all POE approaches, leading tohighly specific results from POE case studies and alack of generalizable conclusions to guide additionalresearch or changes in design (Vischer, 1985).

Some POE approaches—for example, BIU assess-ment—have replaced the emphasis on user satisfactionwith questions that target a more functional evaluationof the work environment. BIU survey questions, forexample, ask respondents to identify their level of com-fort in relation to the specific tasks of their job. Theintent is to shift the user feedback away from personallikes and dislikes toward what might be called an“objective” assessment of the functionality of the work


environment. For example, lighting quality is ratedaccording to the respondent’s task requirements: workat a computer screen, reading print-out or other docu-ments, or appraising forms, colors, and other visuallyoriented tasks. This approach is based on the conceptof “functional comfort”; theoretically, any buildinguser can evaluate functional comfort for any otherperson performing the same tasks in the same envi-ronment.

Organizations that have asked, “Do you like or dis-like . . . ?” or “Are you satisfied or dissatisfied with . . . ?”have found the results too subjective to be useful.Managers fear that these questions raise users’ expec-tations and cause them to expect wide-ranging correc-tive measures and/or gratification of their wishes.Although some POE studies target correcting problemsin the building, many are initiated without a budget ora procedure for follow-up. Moreover, companies oftendesign questionnaires with little regard to the consider-ations necessary for good survey design. Often suchquestionnaires are overly long and detailed, andanalysis of the results is almost always limited to simplefrequencies and percentage calculations. As a result,companies find the data less than useful and concludethat occupant surveys are a waste of time.

Complexity of the Design Process

People who are trained in and perform buildingdesign have difficulty moving beyond design into aPOE once their buildings are occupied and built. Somedesigners evince a general curiosity that generatessome unfocused evaluation activity that gives them asense of the success of their design decisions. The feed-back that the designer acquires rarely goes further thanthe individual seeking the information. Some designerswill spend considerable time on acquiring feedback,whereas others have almost no curiosity, and most aresomewhere in between. Why are design professionalsnot more curious to learn from the positive and nega-tive impacts of their design decisions? I believe theanswer to the question is the complexity of the designprocess, for the following reasons.

The approach of nondesigners to POE is somewhatdifferent from that of designers: the former approachspace as another cultural artifact to be studied usingtraditional social science methods. Some studies seekto orient the space evaluation to the basic design deci-sions and/or criteria. In the words of Inquiry by Design(Zeisel, 1975), each design decision is a hypothesis for

research. Other researchers have taken architects’decisions as hypotheses and tested them with POE data(Cooper, 1973). This approach implies a linear logicaccording to which programming (pre-design informa-tion gathering) leads to design decisions, which lead toconstruction of what has been designed, which in turnleads to POE. Montgomery elaborates on this linearlogic in his introduction to Architects’ People (Ellisand Cuff, 1989). As he points out, this may be a modelfor a rational world, but it is all too clear that the worldof architecture and real estate is anything but rational,that design itself is not rational, and that trying system-atically to link POE with design for those involved inthat process is all but impossible.

What researchers are less aware of, and the designeris painfully aware of, is the irrational nature of designdecisions. Each design decision on a project is influ-enced by the personality traits, role and status, and per-sonal opinions of the individuals involved (client,project manager, architect, contractor, etc.) as well asby the stage reached at the time of making the decision,how involved and informed users are, expectations,budget, and other pressures such as government regu-lations, site constraints, and so forth. The designer’sown design ideas, and how these are communicated,when and to whom, also affect the process. These arebut a few of the factors that mean that building designis not controlled by any one person or agency and thattherefore a clear notion of a project’s design ideas andintentions for the purpose of POE is difficult to identify.

The author’s own experience of highly participatorydesign processes has provided first-hand experience ofthe convoluted, political, and anything-but-lineardecision-making process that causes a building to bewhat it ultimately is. In many cases, no amount ofrational planning and programming can change thelikelihood that once occupied, the use of the spacebegins immediately to change. Sometimes small adjust-ments are made creating incremental change over time,and sometimes the basic assumptions that guided thedesign of the building are dramatically forgotten andthe space is adapted to serve a different purpose.

Given the nature of building design and construc-tion processes, it is unrealistic to expect a designer toseek out feedback on the long-term effectiveness ofdesign decisions on a systematic basis. However, thisis more of a comment on the POE process than on thePOE product, a product whose usefulness cannot bedenied in spite of the complexity of the decisions thatgo into the creation of new physical environments.


Primarily Negative Feedback

Because of the emphasis on building user feedback,much of the information received from POEs is criticalin nature and appears to assign inordinate weight tononfunctional or dysfunctional aspects of a buildingwith little mention of what works. This impression,although false, makes it difficult for information to beshared in a constructive and useful way. The best wayaround this dilemma is for a skilled POE researcher toweigh the importance of the information received.

For example, 10 comments from building usersabout dripping soap dishes in bathrooms, slippery frontsteps, and dust on the work surface cannot be com-pared in importance to one or two comments about poorlighting or a lack of meeting rooms. The former areirritants; they should not carry the weight of an itemthat creates a serious dysfunctionality or impedes usereffectiveness. Similarly, it has frequently been thisauthor’s experience during focus group sessions withusers to listen to 45 minutes of complaints and negativecriticism about their space only to have them comment(once they have got all this off their chests) that theylike the daylight and view from the windows, the spaceis much better than where they used to work, and theylike working in the building!

One of the challenges of POEs going forward is toidentify a reasonable system of informed weighting ofuser feedback so that the data received can be inter-preted according to balanced positive and negative cat-egories.

Complexity of Managing POE Information

The information that results from POE is directed ina number of different directions. In some cases, solu-tions are sought to problems that have been identifiedin a building, and the information is directed to facilitymanagers, building owners, and landlords. In others,the information is directed to designers to help themmake better design decisions on a specific project orgenerically with regard to a building type. In somecases, building users are informed regarding the resultsof a POE in which they were involved, as a way ofinvolving them in planning change and finding ways ofimproving the environment. In yet other cases, theinformation is seen as valuable in itself and dissemi-nated to researchers seeking to understand more aboutthe person-environment relationship. Finally, informa-tion about building systems performance, occupant

functional comfort, operating costs, and adaptation andre-use is directed to stakeholders in the planning,design, construction, and occupancy process who arein a position to make decisions about future buildingprojects.

For each of these, and no doubt other, applicationsof POE information, some thought needs to be given atthe outset to collecting and presenting POE informa-tion in a way that suits the receiver and consumer ofthat information. This means that a clear understandingof the context is necessary and that the POE processshould be designed as a function of contextual con-straints. Key questions to be asked before any POEstudy include the following: Who wants the POE? Howdo they want to use the information? What resourcesare available to gather, analyze, and disseminate theinformation? Who will receive the results, and when?What expectations do stakeholders have of the POEresults?

THE FUTURE OF POE: RECOMMENDATIONS FORAN UNOBTRUSIVE POE PROCESS

The importance of the process used in carrying out aPOE cannot be underestimated; in this author’s opinionit is more important than the method selected and thedata gathered. Once users are involved—as they areonce they are questioned about their use and occupancyof a building—how they are approached, what infor-mation they are given, and the follow-up they experi-ence are all critical stages in the development of therelationship between the occupant and his physicalenvironment. The POE, then, provides an opportunityfor improvement not only of the building and of theenvironment it provides to its users, but also of the wayusers perceive and feel about their territory.

Ideally, one would like to see POE carried out sys-tematically on a wide variety of types of building, butnot before clear objectives and results are identified.At the outset, it is important to clarify the value that thePOE will have for the person or agency carrying it out.If one can identify that the POE has value in the contextin which it is being implemented, then decisions aboutfinancing it, identifying the right things to be studied,and disseminating the results and information to theright people will follow. As the examples in this chaptershow, stakeholders in private sector real estate devel-opment fail to attach value to POE, and even publicagencies—for which the value of POE is apparent—


find that the complexities of the process outweighpotential gains from POE.

In conclusion, it is proposed that a workable POEprocess designed to succeed outside academic circlesincorporate the following steps:

1. A simple, reliable and standardized way shouldbe developed of collecting useful feedback fromoccupants, not on the entirety of their experienceof using the building, but on a few, carefullyselected and identified indicators of environ-mental quality.

2. The indicators selected for measurement shouldbe decided beforehand according to that which ismost relevant to the initiators of the POE and thecontext in which it is implemented.

3. It is necessary to clarify at the outset who arelikely to be the consumers of POE results andtherefore how best to communicate these resultsto them.

4. Consideration should be given to POE techniquesthat avoid direct questioning of users—for example,using instruments, observations, expert walk-throughs, etc.—as well as to refining socialscience techniques to devise reliable and rapidways of questioning building occupants.

5. Efforts to combine instrument data collection andsurveys of building users can be costly, becauselarge amounts of data are generated without yield-ing much additional useful information. Oneapproach is to use the analysis of user responsesto indicate where and when follow-up instrumentmeasures might clarify the nature of the problemsidentified and indicate possible solutions.

6. Users should be well informed regarding the pur-pose of their involvement is providing feedbackand should be made aware in situations whereimmediate correction of problems is not envi-sioned. In fact, it is necessary to recognize thatbuilding users can be “measuring instruments”of environmental quality, rather than only cus-tomers to be served.

7. A decision should be taken at the outset as towhether or not user survey results will be madeavailable to building occupants and, if so, in howmuch detail and for what purpose.

8. Resources for carrying out the POE should bedefined clearly so that data collection and analysisactivities fit into time and budget constraints,however modest.

9. If a questionnaire is given to occupants, it shouldbe designed and analyzed by someone knowl-edgeable in survey research, even if this person isnot involved in the eventual use and applicationof the results.

10. A standardized approach that allows building pro-fessionals (designers, developers, managers) tocollect modest amounts of comparable data froma variety of buildings to analyze on a comparativebasis is likely to be more useful than a detailedone-off case study in most situations.

11. Public agencies should examine the possibility ofsetting up test POEs on a demonstration basis, todevelop POE techniques, to demonstrate value,and to determine the best ways of making POErelevant to the building industry.

ABOUT THE AUTHOR

Jacqueline Vischer is an environmental psychologist.She is currently professor and director of the interiordesign program at the University of Montreal. She hasconsulting experience in architecture and planningprojects in the United States and Canada. As principalin her own consulting firm in Vancouver, Canada,Dr. Vischer and her staff undertook contract researchfrom government agencies in residential planning andevaluation, institutional programming, and policyanalysis. Dr. Vischer then spent five years developingbuilding performance studies of office buildings forPublic Works Canada in Ottawa, projects from whichthe building-in-use assessment system emerged. Shethen undertook the design and implementation of apost-occupancy evaluation program for public build-ings owned and operated by the State of Massachusetts’Division of Capital Planning. In 1989 Dr. Vischerstarted the Institute For Building Science, whichbecame Buildings-In-Use in 1990 and opened itsMontreal office, Bâtiments-en-Usage, in 1991.Dr. Vischer has held positions as lecturer and instruc-tor at the McGill University School of Urban Planning,University of British Columbia, and Harvard Uni-versity’s School of Design. She is a member of theEnvironmental Design Research Association, theAmerican Society for Heating Refrigeration and AirConditioning Engineers, the Montréal MetropolitanEnergy Forum, and the International Facilities Man-agement Association. Dr. Vischer holds a bachelor ofarts in psychology from the University of California,Berkeley, and a master of arts in psychology from the


University of Wales Institute of Science and Technol-ogy, and a Ph.D. in architecture from the University ofCalifornia, Berkeley.

REFERENCESBoston Globe. (1987). 30 June.Brill, M., Margulis, S.M., and Konar, E. (1985). Using Office Design to

Increase Productivity (2 vols.). Buffalo, N.Y. : BOSTI andWestinghouse Furniture Systems.

Business Week. The new workplace. (1996). April 29, pp.107-117.Centre scientifique et technique du bâtiment (1990). Améliorer

l’architecture et la vie quotidienne dans les bâtiments publics. Paris:Plan construction et architecture, Ministère des équipements, dulogement, des transports et de l’espace.

Cooper, C. (1973). Comparison Between Architects’ Intentions and Resi-dents’ Reactions, Saint Francis Place San Francisco. Berkeley, Calif:Center for Environmental Structure.

Dilani, A. (ed.) (2000). Proceedings of the 3rd International Conference onHealth and Design. Stockholm: University of Stockholm.

Dillon, R. and Vischer, J. (1988). The Building in-Use Assessment Method-ology (2 volumes). Ottawa: Public Works Canada.

Ellis, W.R, and Cuff, D. (1989). Architects’ People. New York: OxfordUniversity Press.

Farbstein, J., and Kantrowitz, M. (1989). Post-occupancy evaluation andorganizational development: the experience of the United States PostalService. In: Building Evaluation. Preiser, W. (ed.). New York: PlenumPress, p. 327.

Friedman, A., Zimring, C., and Zube, E. (1979). Environmental DesignEvaluation. New York: Plenum Press.

Joiner, D., and Ellis, P. (1989). Making POE work in an organization. In:Preiser, W. (ed.) Building Evaluation. New York: Plenum Press, p. 299.

Marans, R., and Spreckelmeyer, K. (1981). Evaluating Built Environments:A Behavioral Approach. University of Michigan, Survey ResearchCenter and Architectural Research Laboratory.

Public Works Canada (1983). Stage One in the Development of Total Build-ing Performance (12 volumes). Ottawa: Public Works Canada, Archi-tectural and Building Sciences.

Ventre, F. (1988). Sampling building performance. Paper presented atFacilities 2000 Symposium, Grand Rapids, Mich.

Vischer, J. (1985). The adaptation and control model of user needs inhousing. Journal of Environmental Psychology 5:287-298.

Vischer, J. (1989). Environmental Quality in Offices. New York: VanNostrand Reinhold.

Vischer, J. (1993). Using occupancy feedback to monitor indoor air quality.ASHRAE Transactions 99 (Pt.2).

Vischer, J. (1996). Workspace Strategies: Environment as a Tool for Work.New York: Chapman and Hall.

Zeisel, J. (1975). Inquiry by Design. New York: Brooks-Cole.Zeisel, J. (in press). Inquiry by Design, 2nd edition. New York: Cambridge

University Press.

35

4

Post-Occupancy Evaluation Processes inSix Federal Agencies

The federal government is the largest owner offacilities in the United States. More than 30 individualfederal agencies own, use, and acquire facilities to sup-port agency missions and programs. Some federalagencies conduct post-occupancy evaluation (POE)and lessons-learned programs as ways to improve cus-tomer satisfaction, to increase building quality and per-formance, and to facilitate organizational learning. Thischapter provides information about POE processes insix federal agencies: the U.S. Air Force, Office of theCivil Engineer; the General Services Administration,Public Buildings Service (PBS); the Department of theInterior, National Park Service (NPS); the U.S. Navy,Naval Facilities Engineering Command (NAVFAC); theU.S. Department of State, Office of Overseas BuildingsOperations (OBO); and the U.S. Postal Service (USPS).

The six agencies are sponsors of the Federal Facili-ties Council and volunteered to participate in the study.The information was gathered through a questionnaireand telephone interviews conducted by Krista Waitz ofKwaitz Consulting. National Research Council staffwrote the summary of findings and descriptions of POEprograms.

The study design was not a scientific one, nor was itbased on random sampling. Thus, the information pro-vided should not be generalized. The remainder of thischapter contains information about the survey ques-tions, a summary of findings, and descriptions of thePOE programs in the six agencies.

SURVEY QUESTIONS

In January 2001, a questionnaire was designed andissued to six sponsor agencies of the Federal Facilities

Council who volunteered to provide information abouttheir post-occupancy evaluation processes. In somecases, more than one person in the agency respondedand the responses were combined and reconciled. Theagency representatives were asked to respond to thefollowing questions:

1. Approximately what year did your agency estab-lish a post-occupancy evaluation program? Onaverage, approximately how many POEs havebeen conducted by your agency in each of the lastfive fiscal years?

2. What were the driving factors for establishing aPOE program?

3. What is the focus of the POE information-gather-ing process (e.g., user satisfaction, achievementof design objectives, building performance,other)?

4. What are the expectations, goals, and objectivesfor the program? Have they been achieved? Havethere been unanticipated results?

5. How is the information gathered through POEstied into the feedback loop (lessons learned) forplanning, programming, and capital asset man-agement?

6. Is the information gathered through POEs used inreal estate decision-making and capital asset man-agement? If yes, please note what information isused, how, and when it is used. If no, why not?

7. Is it your agency’s policy to conduct POEs for allbuildings or for selected facilities? What are thecriteria for determining whether a POE will beconducted?


8. What do you consider to be barriers to conduct-ing successful POEs?

9. Who is typically involved in conducting a POE interms of in-house personnel and consultants?(Please list positions or types of skills involvednot individuals.) What types of technologies areused?

10. What is the estimated cost in time and dollars forconducting a typical POE?

11. What data collection methods, technologies, andsurvey forms have been used over the life of theprogram? Please provide copies of survey formsthat have been used.

12. To what extent does your agency make use of datamanagement systems, Internet tools, or otherinformation technology applications to shareinformation and disseminate results of POEs?

SUMMARY OF FINDINGS

Establishment of POE Program, Timing of Surveys,and Number Conducted

Each of the six agencies studied had had a POEprogram in place at least since the 1980s. The POEprograms of the PBS, NPS, and NAVFAC were beingrestructured to meet new objectives, and the results ofthe reorganized programs were not yet available.

The number of POEs conducted annually, on aver-age, ranged from less than 1 to 30. Post-occupancyevaluations are typically performed within 4 to 24months following occupancy of a new or renovatedfacility and are performed only once for an individualbuilding.

Focus of POE Programs

Each of the six agencies used POEs to determineclient or user satisfaction at some level, but it also usedthem to fulfill other objectives. These objectivesincluded determining building defects within the con-struction warranty period, supporting design and con-struction criteria, supporting performance measures forasset management, evaluating construction inspectors,lowering facility life-cycle costs by identifying designerrors that could lead to increased maintenance andoperation costs, clarifying design objectives, improvingbuilding performance, and supporting corporate salesand image objectives. The restructured programs inNAVFAC and PBS are focused on developing metrics

for client satisfaction and for management-relatedissues.

POE Process and Technologies

No two agencies use the same process or tools forconducting POEs and capturing lessons, although someshare common elements. The National Park Service isin the process of developing new procedures and toolsfor conducting POEs and sharing lessons learned. Cur-rently, at the NAVFAC, an independent agent remotefrom the designer of record conducts the POE using astatistically based questionnaire; a focus group discus-sion is then conducted to summarize the results of thesurvey. The NAVFAC criteria office administerssurvey results. The questionnaire is being modified sothat it can be administered from field agencies and canbe completed on-line or downloaded from the Web.The survey documents and results can be downloadedfrom a database on the Web.

The Air Force, in contrast, uses a questionnaireadministered by a staff team; feedback is given prima-rily to the construction agent, although the Air Forceplans to also share the results with users, the base civilengineer, and the major command.

The Office of Overseas Buildings Operations usespre-trip user questionnaires, on-site interviews, andfacilitated town meetings to gather the information,which is then summarized in a formal report. OBO’sPOE is conducted by a multidisciplinary in-house teamthat is customized to address known deficiencies.

The PBS performance measures-oriented approachuses a set of questionnaires developed in cooperationwith the Center for the Built Environment at the Uni-versity of California at Berkeley. The questionnairesare being designed to be administered over the Web.The lessons learned will provide input to the ongoingperformance measures program of the Office of Busi-ness Performance.

The U.S. Postal Service uses two levels of surveys.The first, a basic questionnaire that can be completedin about an hour by the administrative service officerand the postmaster of a new facility, is required for allnew construction. For larger, more complex projects,POEs are conducted over two to four days using amultidisciplinary team. Both types of surveys use elec-tronic questionnaires in Microsoft Excel. The informa-tion gathered is sent directly to the staff maintainingthe agency’s design standards.

POST-OCCUPANCY EVALUATION PROCESSES IN SIX FEDERAL AGENCIES 37

How POE Information Is Used

Information obtained from POE programs has beenused by the OBO, NAVFAC, PBS, and USPS in sup-port of their design criteria and guidelines. The NPSand the Air Force programs anticipate using POEresults for the support of design criteria, among otherobjectives. To date, none of the six agencies reportedthat POE information was used directly in future realestate decision-making and capital asset management,although PBS’s POE program was being restructuredwith those objectives in mind.

Barriers

A number of barriers to more effective use of POEsand lessons-learned programs were identified. Thesecould be categorized generally as resources, feedback,and participation and commitment.

Resources

Several agencies noted it was difficult to obtain orearmark the funding needed to conduct POEs regard-less of whether the POEs were to be conducted usingconsultants or in-house staff. In some cases, in-housestaff may not be available to conduct the POE or maynot have the technical skills needed for quality results.

Feedback

Because POEs often focus on identifying deficien-cies, they risk becoming instruments to focus or deflectblame for unsatisfactory results. One agency cited theconcern by senior executives that lessons learned maybe considered a weakness by Congress or the InspectorGeneral. Other agencies noted that conducting a focusgroup to achieve consensus about the cause of failureswithout judgmental discussion can be difficult, and asa consequence, they may be reluctant to do so.

Participation and Commitment

One agency noted that because of constructionschedule constraints, staffs may be focused on futureprojects or those under visible construction. Thus, oncea project has been completed and occupied, items suchas financial closeout, construction as-builts, and POEsmay not be a high-priority item and may not receiveadequate oversight or attention. For programs adminis-

tered through a headquarters’ office, there may be alack of field-level attentiveness to the process. Obtain-ing the clients’ attention to ensure adequate participa-tion in the survey or getting people not originallyinvolved with the project to participate in a survey wasalso identified as a barrier. Organizational structurescan also create barriers when responsibilities areassigned such that POE administration and databasedevelopment require interoffice collaboration.

Costs

The costs reported to conduct POEs ranged from$1,800 for a simple standard questionnaire that couldbe completed in one hour to $90,000 for an in-depthanalysis including several days of interviews, multi-disciplinary teams, site visits, and writing up reports.The costs did not include implementation of anychanges resulting from a POE study. Costs per squarefoot of space evaluated were not available. Other vari-ables accounting for the range of costs includedwhether the facilities were located in the United Statesor abroad, whether in-house staff or consultants wereused, and how the resulting information was packagedand distributed.

DESCRIPTIONS OF POE PROGRAMS

National Park Service, Department of the Interior

In the mid-1980s the National Park Service com-pleted formulation and development of an extensivePOE program; however, due to changes in staff anddownsizing of the NPS central design office, theprogram was not fully executed. In 1998, the POE pro-gram was reinstituted as part of the business practicesfor the NPS central design office. At that time, it wasdecided to reestablish the POE program for threereasons: (1) to create a feedback loop that would allowdesigners to interact with facility users and learn iffacilities were meeting the needs of the users; it alsoserved as an opportunity for users to assess their origi-nal requirements and determine if they had adequatelyidentified their needs; (2) to evaluate the effectivenessof consultant construction inspectors who had recentlyreplaced all NPS construction inspectors; and (3) toimprove long-term facility life-cycle ratio costs byidentifying any design errors that could lead toincreased maintenance or operational costs.


The focus of the NPS POE program is to improveuser satisfaction, building performance, and designerefficiency. The central design offices for the NPS seekto retain highly talented designers over a long-termcareer. By building relationships with facility users,designers better understand user needs and anticipaterequests, allowing for more efficient use of designfunds. The POE program seeks to build on this type ofrelationship through open communications and on-sitereview of completed construction projects. The NPSexpects the value-added component of the POE will bein lessons learned and improved design efficiency. TheNPS noted that maintenance and operations costs forfacilities are escalating annually, and if the POE pro-vides information for future designs that lead toimproved maintenance or operations, the payback willbe dramatic. In addition, if the POE provides feedbackon products or techniques that improve user satisfac-tion, reduce maintenance, or improve operations, thisinformation can be shared with designers to reduce thecost of design development.

Data collection tools and the forum for sharing infor-mation are still under development. However, the POEprogram is envisioned to include an evaluation formand follow-up meetings approximately six months toone year after completion of construction. The POEmeetings will include the users, project manager, anddesign team captain. After completing the evaluation,the materials will be shared within the project manage-ment and design divisions. The results of the POE willalso be placed in the central technical information filesand may be accessed by various levels of project man-agers, designers, and other technical staff.

The NPS anticipates its POE teams will consist of(1) users, including park superintendent, chief of main-tenance, park rangers, park interpreters, and adminis-trative staff; (2) project managers, a multidisciplinarygroup consisting of park planners, architects, landscapearchitects, and various disciplines of engineers—thetechnical expertise will vary by project; and (3) designteam members. Generally a team captain for a buildingproject would be a senior architect; for a road construc-tion project, a senior civil engineer or senior landscapearchitect would attend the POE; and for a utility project,a senior civil engineer would attend. Depending on thecomplexity of the project, other members attending aPOE meeting could include a mechanical engineer oran electrical engineer.

Naval Facilities Engineering Command,Department of the Navy

The Naval Facilities Engineering Command hasconducted POEs since the 1960s. In its original form,NAVFAC’s POE program had an instructional baseand was conducted by a project team after constructionto identify lessons learned. In 1997, a new statisticallybased concept was initiated. The goal is to establish astatistical basis from which NAVFAC can measureagency improvement and work toward continuousimprovement by implementing process changes andmodifying design criteria. The POE program is intendedto apply to each completed facility within its warrantyperiod. On average, 20 POEs have been conducted ineach of the last five fiscal years.

There were two driving factors for restructuring thePOE program at NAVFAC. The first was a publishedDepartment of Defense-level survey of occupantsatisfaction with their facilities. The second was aNAVFAC headquarters’ management initiative to cre-ate metrics measuring how, or if, the agency exceededclient expectations. The agency focuses on improvingclient satisfaction, determining where its product linesor processes give rise to client dissatisfaction andimproving the individual facility on which the surveyis conducted. NAVFAC is issuing policy that willrequire a survey of all facilities within 6-10 months ofbuilding occupancy. The focus of the POE informa-tion-gathering process is to measure user satisfactionfrom the perspectives of the building owner, the cus-tomers (student, family occupant, day care familymember, etc.), the building occupant at the workinglevel, and the staff maintaining the facility. Clientsatisfaction at the user level is measured whether or notthe client was involved in the planning, design, or con-struction phases. The measuring process includes asurvey of participants after which a focus group dis-cussion is conducted to summarize the positive andnegative aspects of the facility.

Currently, all POE surveys are conducted by an“independent agent” remote from the designer ofrecord, typically a consultant. The NAVFAC criteriaoffice administers the surveys. The criteria managerfor the facility type identifies criteria issues andaction(s) to be initiated and implements interim guid-ance to the organization when required. The survey willbe used in developing and modifying planning anddesign criteria. The data will be reviewed annually todetermine process improvement needs. NAVFAC is


working to determine how the survey will interfacewith its knowledge management system, combining toyield a single lessons-learned concept for the organi-zation.

When the POE program process began, NAVFACused a checklist format focusing only on design andconstruction. The survey was changed to accommodateall of the agency’s processes (i.e., planning, design,construction, and maintenance turnover). Additionalminor changes are anticipated to better assess safetyand procurement issues. A copy of the survey is con-tained in Appendix D.

The database is being modified to Web-enable thesurvey content and to create a field-managed site asopposed to a headquarters central database. In addi-tion, the database is being made integral to corporateORACLE-based management systems. It will drawinformation from the management system and alertassigned individuals when it is time for the survey.

Office of the Civil Engineer, U.S. Air Force

The focus of the Air Force POE information-gathering process is user satisfaction, achievement ofdesign objectives, and improved building performance.The purpose of the POE is to note all defective work,report construction deficiencies to the constructionagent for correction by the contractor, and documentproblems or mistakes made during design for use aslessons learned on similar projects.

The POE is conducted by a staff team using a ques-tionnaire. The Air Force plans to share the results ofthe POE with the construction agent, the user, the basecivil engineer, and the major command for use in anyfuture designs and for incorporation into Air Forcedesign standards. It is Air Force policy for a POE to beaccomplished sometime during the ninth to eleventhmonth following beneficial occupancy (acceptance ofthe facility by the user agency).

Office of Overseas Buildings Operations,Department of State

At the Office of Overseas Buildings Operations(formerly Foreign Buildings Operations) of the U.S.Department of State, lessons learned and design or con-struction alerts have been issued since 1985. The driv-ing factors for establishing a POE program were theconcern for user satisfaction, comfort, and safety and ageneral desire to capture best practices. The focus of

the POE information-gathering process is on usersatisfaction, achievement of design objectives, andbuilding performance, including interior flexibility andfunctionality.

The POE methodology followed was adapted fromthe U.S. Postal Service. Over the life of the POEprogram, occupant surveys, on-site interviews, andfacilitated town meetings have been used for data col-lection. The first step in the POE process is to send anoccupant survey to an overseas post. The preliminaryresults from the occupant survey are used to determinewhat disciplines should be represented on the multi-disciplinary team that will be conducting the POE; thus,the team is customized to address known deficiencies.Typically, architects; electrical, mechanical, and struc-tural engineers; facility maintenance; and securityspecialists are involved in conducting POEs. Once atthe site, the team conducts a walk-through followed upby interviews with occupants. At the conclusion of thesite visit, members of the team reconvene to discusstheir observations and to generate recommendations fora report. Due to constrained resources, a POE may con-sist solely of the occupant survey.

One result of the POE program has been the revisionof design guidance on such topics as roofs, elevators,and Ambassadors’ residences. POE results were alsoused for developing a serviceability demand profile forgeneric embassy office buildings slated for design andin design guidelines for future embassies being acquiredunder a specialized procurement process.

Public Buildings Service,General Services Administration

The Public Buildings Service (PBS) of the GeneralServices Administration (GSA) first instituted a POEprogram oriented toward design and construction crite-ria development in 1977. In-house technical experts,including an environmental psychologist, architects,and engineers, conducted the surveys. The programwas curtailed in 1982 due to a reduction in staff. PBS’POE program was reinstituted in 1986, using contractorsupport. Between 1986 and 2000, the PBS completedapproximately 30 POEs for a variety of projects,including courthouses, office buildings, U.S. borderstations, major renovations, and historic restorations.In 2000, the PBS restructured its POE program to focuson performance measures for asset management. Boththe design criteria-oriented POE program and theperformance measure-oriented POE program were


intended to provide an information stream that wouldinform program managers, criteria managers, andproject managers about design- and delivery-relatedproblems and associated best practices.

Criteria–Based POEs

Criteria-based POEs were developed to providetechnology- and procedure-based feedback to those inPBS’s central office responsible for national programand design criteria direction. Equally important werethe perceived benefits of offering those same lessons tothe delivery teams responsible for new projects. Theprimary focus was on building systems evaluation,client satisfaction surveys, and interviews with majorclient agencies. The building systems evaluationsincluded functionality issues as well as an overview ofhow well the building complied with design criteria.The surveys were customized for different types ofbuildings and project delivery systems, including officebuildings, courthouses, border stations, and lease-buildor design-build projects. (An example is included inAppendix D.) Major lessons from the POE processrelated to long-term building maintainability, buildingfunctionality, client needs, and property manager andasset management needs. Specific issues concernedenergy efficiency; indoor air quality; heating, ventila-tion, air conditioning, and electrical systems; thermalcomfort; design of loading docks; access to equipment;window washing; and accessibility for the physicallydisabled. The criteria were constantly revised to incor-porate lessons from POEs.

From 1986-2000, criteria-based POEs typically in-volved a team of five to six design-related disciplines,using outside architect-engineer professional services.PBS central office coordination and involvement wereprovided through site visits, access coordination, andreport critiques. A report and a 30-minute video werecreated and distributed to each of GSA’s field offices(11 regions) and to senior-level management at head-quarters.

The tieback to lessons learned was direct: the peoplewho conducted the POEs also developed the criteria.However, because the POE reports were voluminousand oriented toward detailed evaluations of technologyapplications, getting project managers and designers toread and adhere to findings was a challenge. Thus, vari-ous forms of information exchange media were appliedto mitigate this problem, including condensed video-tapes and interactive “lessons-learned” compact disks

(CDs). A compendium of lessons was distributed everythree years. In 1998, PBS prepared a Compendium ofLessons Learned CD-ROM that was widely distributed,and a DVD (Lessons Learned, Volume 2) is being pre-pared for mass distribution. The POEs are also in theconstruction criteria database of the National Institutefor Building Science.

A second program that resulted from the POE les-sons is HVAC (heating, ventilation, air conditioning)Excellence in Federal Buildings. PBS held numerousworkshops that included staff, architects, engineers,and representatives from professional societies andtechnical organizations as part of an awareness pro-gram to highlight HVAC issues.

Performance Measure-Oriented POEs

The driving factor behind the shift to a performancemeasure-oriented POE program was PBS senior man-agement’s desire to evaluate how well PBS’s assets areachieving their objectives on a project and programbasis. Performance measure-oriented POEs are beingpursued to help indicate whether delivery practices andcriteria are effective and to identify systemic problems,whereby specialty studies could be pursued.

The focus of this POE information-gathering processis financial asset assessment. A set of extensive ques-tionnaires is used in an attempt to identify customersatisfaction with various building components orfeatures. Different questionnaires are directed to dif-ferent key personnel, including operating staff anddesign-delivery team members. The measures areintended to help determine if GSA is meeting a numberof key management indicators including comparisonof construction “pro forma” with final pro forma, main-tenance and cleaning costs benchmarked againstnational standards, utility costs, sustainability, energyusage against FY 2010 goals, accessibility for thephysically disabled, and client satisfaction.

Support for questionnaire development and databasemanagement is currently provided by the Center forthe Built Environment, within the University of Cali-fornia at Berkeley. Questionnaire delivery and assess-ments are being coordinated by senior architects andengineers within the PBS Office of Business Perfor-mance. PBS is currently designing tenant, operationsand maintenance, and design and construction surveytools that can be administered over the Web.

The goal is to perform a POE for every Congres-sionally approved new building one year after full


occupancy. Once the performance measure-orientedPOE program is fully implemented, 10-20 POEs willlikely be completed each year. The tieback from theongoing performance measure POE program will gofrom PBS’s Office of Business Performance to otherappropriate offices within the agency.

U.S. Postal Service

The U.S. Postal Service established its POE programin 1986. The driving force behind the establishment ofthe POE program was the desire to improve the plan-ning, design, and construction of future facilities. Thefocus of the POE information-gathering process is usersatisfaction (customers and employees), clarification ofdesign objectives, achievement of design objectives,building performance relative to technical systems suchas cooling and lighting, and supporting corporate salesand image objectives and economics. On average, theagency has conducted approximately 30 POEs in eachof last five fiscal years.

The first POE application was an effort to standard-ize the hundreds of Postal Service retail spaces pro-duced each year. Appropriate design was found to be amuch more powerful factor in customer satisfactionthan had been anticipated. Also, building image was a

much more powerful support for overall corporateidentity than previously realized.

Two levels of POEs are used currently. A basic POE(completing the questionnaire) is required for all newconstruction and for owned facilities greater than 9,000square feet within four and six months of occupancy.The administrative service office manager and the post-master complete the basic POE questionnaire, whichtypically takes 30 minutes to an hour. An example isincluded in Appendix D.

More extensive POEs are conducted on largerprojects (more than 30,000 square feet) or other specialprojects. These POEs involve architectural or engi-neering firms (including environmental psychologistsas consultants) and are conducted over a period of twoto four days. Customers and employees are inter-viewed, and extensive lighting and HVAC data aregathered.

The results of POEs go directly to the staff main-taining the Postal Service Building Design Standards.The information gathered through the POE process isnot used in real estate decision-making or capital assetmanagement; however it is used in planning, design,and construction decisions. Real estate decisions areaffected only as site-planning criteria are modified.

42

5

Post-Occupancy Evaluations and Organizational Learning1

Craig Zimring, Ph.D., Georgia Institute of TechnologyThierry Rosenheck, Office of Overseas Buildings Operations, U.S. Department of State

1For their generous and thoughtful input we would like to thankStephan Castellanos, Dennis Dunne, Gerald Thacker, LyndaStanley, Polly Welch and Richard Wener.

Federal building delivery organizations face intensepressures. Not only must they provide buildings ontime and within budget, but they have increaseddemands. They are called on to deliver buildings thatare better: more sustainable, accessible, maintainable,responsive to customer needs, capable of improvingcustomer productivity, and safer. In many cases, theymust achieve these goals with fewer staff.

Some organizations have faced these pressuresproactively, by creating formal processes and culturalchanges that make their own organizational learningmore effective. In this chapter we adopt the approachto organizational learning of Argyris (1992a), Huber(1991) and others. We mean that organizations are ableto constantly improve the ways in which they operateunder routine conditions, and they are able to respondto change quickly and effectively when needed(Argyris, 1992a). Learning is “organizational” if it isabout the core mission of the organization and isinfused through the organization rather than residing ina few individuals. More simply, in the words of DennisDunne, chief deputy director for California’s Depart-ment of General Services, they “get it right the secondor third time rather than the seventh or eighth.” Bybeing more systematic about assessing the impact ofdecisions and being able to use this assessment in futuredecision-making, building delivery organizations areable to reduce the time and cost to deliver buildingsand increase their quality.

Some of the best models come from private sectororganizations. For example, Disney evaluates every-thing it does and has been doing so since the 1970s.Disney has at least three evaluation programs and threecorresponding databases: (1) Disney tracks the perfor-mance of materials and equipment and records the find-ings in a technical database. (2) Guest services staffmembers interview guests about facilities and services,focusing on predictors of Disney’s key business driver:the intention of the customer to return. (3) A 40-personindustrial engineering team conducts continuousresearch aimed at refining programming guidelines andrules of thumb. The industrial engineering teamexplores optimal conditions: What is the visitor flowfor a given street width when Main Street feels pleas-antly crowded but not oppressive? When are gift shopsmost productive? This research allows Disney to makedirect links between “inputs” such as the proposednumber of people entering the gates and “outputs” suchas the width of Main Street.

The Disney databases are not formally linkedtogether but are used extensively during design andrenovation projects. They have been so effective thatthe senior industrial engineer works as a coequal withthe “Imagineering” project manager during the pro-gramming of major new projects.

Disney is a rare example. It uses an evaluation pro-gram to do the key processes that organizational learn-ing theorists argue are key to organizational learning(Huber, 1991):

1. monitoring changes in the internal and externalbusiness environment,

POST-OCCUPANCY EVALUATIONS AND ORGANIZATIONAL LEARNING 43

2. establishing performance goals based on internaland external influences,

3. assessing performance,4. interpreting and discussing the implications of re-

sults,5. consolidating results into an organizational

memory,6. widely distributing findings and conclusions,7. creating a culture that allows the organization to

take action on the results,8. taking action based on organizational learning.

Post-occupancy evaluation (POE) practicefocuses mostly on individual project support andanalysis rather than on lessons-learned. AlthoughPOE potentially provides a methodology for all of theseprocesses, POE practice has historically had a morenarrow focus on assessing performance and interpret-ing results. POE has often been used as a methodologyaimed at assessing specific cases, while the other pro-cesses are seen as part of strategic business planning.Even when evaluators have been able to create data-bases of findings, they have often been used to bench-mark single cases rather than to develop more generalconclusions.

Structured organizational learning is difficult. Itrequires the will to collect data about performance andthe time to interpret and draw conclusions from thedata. More fundamentally, learning involves risk andchange. Learning exposes mistakes that allow improve-ment but most organizations do not reward exposingshortcomings. Learning brings change and organiza-tions are usually better at trying to ensure stability thanat supporting change.

In this chapter, we explore how a variety of publicagencies and some private ones have used POE suc-cessfully for organizational learning. We discuss the“lessons-learned” role of evaluation rather than theproject support and analysis role. We have examinedwritten materials from 18 POE programs and inter-viewed participants wherever possible. We exploredwhether POE-based organizational learning appearedto be going on, whether the organizations had estab-lished support for learning, and the nature of thelearning.

Did POE-Enabled Organizational Learning Occur?

For example, in looking for evidence of organiza-tional learning we asked the following questions:

• Are participants in building projects, includinginternal project managers, consultants, and clients,aware of POEs or POE results, either from per-sonal participation or from written results?

• If so, were POE results consciously used indecision-making about buildings? For example,are they used for programming, planning, design,construction, and facilities management?

• Can we see evidence that POE results are part ofreflection and discussions about how to do a goodjob, among peers and with supervisors?

• Are POE results consciously used to refine pro-cesses for delivering buildings in terms of eitherformal process reflected in manuals or informalrules of thumb and customs?

• Are people who make policy about buildings,such as policy directives, design guidelines, andspecifications, aware of POEs?

• If so, is POE explicitly used in formulatingpolicy?

Were the Conditions for OrganizationalLearning Present?

As we attempted to document organizational learn-ing we were trying to understand the conditions thatfoster or thwart it:

• Does the organization have an infrastructure forlearning? For example, are results from POEsconsolidated in some way, such as in reports ordatabases? Is this consolidated information dis-tributed, either internally or to consultants or thepublic?

• Is there a mechanism for ensuring that this infor-mation is kept current?

• If lessons are made available, do they support thekinds of decisions that are made by the organiza-tion? Are they likely to seem authentic andimportant to decision-makers? Are the implica-tions of results made clear, or do busy decision-makers need to make translations between resultsand their needs?

• Are there incentives for accessing the data, usingthe results, and contributing to the lessons-learnedknowledge base? For example, are internal staffor consultants evaluated on use of lessons-learned? Are they rewarded in some way for par-ticipation? Are consultants rewarded for partici-


pation? Are they rewarded for good performanceas judged by the POE?

• Are there disincentives for participating in lessons-learned programs? If an innovative initiativereceives a negative evaluation, is it treated as anopportunity for organizational learning or as apersonal failure?

• Is there a perception of high-level support for thelessons-learned program? Many organizationscreate frequent new initiatives, and seasoned staffoften perceive these as the management “infatua-tion du jour”: wait a day and it will change.

How Can Organizations Develop UsefulLearning Content?

We also assessed the content of the organizationalknowledge. We asked the following:

• Has the organization produced a shared view ofwhat makes a good building, in terms of eitherprocess or product? For example, has the organi-zation been clear about key design and program-ming decisions and about how these decisionslink to the client’s needs? Are these contributedto by POE?

• Has the organization created an organizationalmemory of significant precedents? Are these pre-cedents described, analyzed, or evaluated inmeaningful ways?

• Is this view tested and refined through POE orsimilar processes?

In this chapter, we briefly report our findings andanalysis. We discuss four topics:

1. What is post-occupancy evaluation? What is itshistory, and how has this contributed both to itspotential for and difficulties in achieving organi-zational learning?

2. Do organizations do POE-enabled organizationallearning?

3. How have organizations created the appropriateconditions for learning through POE?

4. How have they created a knowledge base forbuilding delivery and management?

BRIEF INTRODUCTION TOPOST-OCCUPANCY EVALUATION

Post-occupancy evaluation grew out of the extra-ordinary confluence of interests among social scien-tists, designers, and planners in the 1960s and 1970s(see, for example, Friedmann et al., 1978; Shibley,1982; Preiser, et al., 1988). Early POE researchers werestrongly interested in understanding the experience ofbuilding users and in representing the “nonpaying”client (Zeisel, 1975). Many early POEs were conductedby academics focusing on settings that were accessibleto them, such as housing, college dormitories, and resi-dential institutions (Preiser, 1994).

During the 1980s, many large public agenciesdeveloped more formal processes to manage informa-tion and decisions in their building delivery processes.As planning, facilities programming, design review,and value engineering became more structured, agen-cies such as Public Works Canada and the U.S. PostalService added building evaluation as a further step ingathering and managing information about buildings(Kantrowitz and Farbstein, 1996).

This growth of POE occurred while politicians andpolicy analysts were advocating the evaluation ofpublic programs more generally. Campbell and manyothers had been arguing at least since the 1960s thatpublic programs could be treated as social experimentsand that rational, technical means could contribute to,or even replace, messier political decision-making(Campbell, 1999). A similar argument was applied toPOE: statements of expected performance could beviewed as hypotheses that POE could test (Preiser etal., 1988).

The term post-occupancy evaluation was intendedto reflect that assessment takes place after the clienthad taken occupancy of a building; this was in directcontrast to some design competitions where completedbuildings were disqualified from consideration or toother kinds of assessment such as “value engineering”that reviewed plans before construction. Some earlydescriptions focused on POE as a stand-alone practiceaimed at understanding building performance from theusers’ perspectives. Some methodologists have advo-cated the development of different kinds of POEs, withdifferent levels of activity and resource requirements(Friedmann et al., 1978; Preiser et al., 1988). Forexample, Preiser advocated three levels of POE: briefindicative studies; more detailed investigative POEs;and diagnostic studies aimed at correlating environ-


mental measures with subjective user responses(Preiser, 1994). Whereas there was little agreementabout specific methods and goals, most early POEsfocused on systematically assessing human response tobuildings and other designed spaces, using methodssuch as questionnaires, interviews, and observation,and sometimes linking these to physical assessment(Zimring, 1988).

Over the years, many theorists and practitioners havegrown uncomfortable with the term POE; it seems toemphasize evaluation done at a single point in the pro-cess. Friedmann et al. (1978) proposed the term “envi-ronmental design evaluation.” Other researchers andpractitioners have suggested terms such as “environ-mental audits” or “building-in-use assessment”(Vischer, 1996). More recently, “building evaluation”and “building performance evaluation” have been pro-posed (Baird et al., 1996). Nonetheless, for historicalreasons the term post-occupancy evaluation remainscommon, and we use it in this chapter for clarity.

Other discussions of evaluation emphasized theimportance of embedding POE in a broader program ofuser-based programming, discussion, and design guidedevelopment, proposing terms such as “pre-occupancyevaluation” (Bechtel, 2000), “process architecture”(Horgen et al., 1999), and “placemaking” (Schneeklothand Shibley, 1995). As early as the 1970s, the ArmyCorps of Engineers conducted an ambitious programof user-based programming and evaluation that resultedin some 19 design guides for facilities ranging fromdrama and music centers to barracks and military policestations (Schneekloth and Shibley, 1995; Shibley,1982, 1985). More recently, POE has been seen as partof a spectrum of practices aimed at understandingdesign criteria, predicting the effectiveness of emerg-ing designs, reviewing completed designs, and support-ing building activation and facilities management(Preiser and Schramm, 1997). With growing concernsabout health and sustainability, several programs havealso linked user response to the physical performanceof buildings, such as energy performance (Bordass andLeaman, 1997; Cohen et al., 1996; Leaman et al., 1995)or indoor air quality (Raw, 1995, 2001).

POE methodologists and practitioners have identi-fied several potential benefits of POE (Friedmann etal., 1978; McLaughlin, 1997; Preiser et al., 1988;Zimring, 1981):

• A POE aids communications among stakeholderssuch as designers, clients, end users, and others.

• It creates mechanisms for quality monitoring,similar to using student testing to identify under-performing schools, where decision-makers arenotified when a building does not reach a givenstandard.

• It supports fine-tuning, settling-in, and renovationof existing settings.

• It provides data that inform specific future deci-sions.

• It supports the improvement of building deliveryand facility management processes.

• It supports development of policy as reflected indesign and planning guides.

• It accelerates organizational learning by allowingdecision-makers to build on successes and notrepeat failures.

This chapter focuses primarily on the use of POE forimproving organizational learning.

DO ORGANIZATIONS DO POE-ENABLEDORGANIZATIONAL LEARNING?

As discussed above, we reviewed materials fromsome 18 organizations that are currently doing POEsor have done so in the past. In looking at organizationsthat have active POE programs, we found that membersof project teams, including project managers, consult-ants, and clients, tend not to be aware of POEs, unlessa special evaluation has been conducted to address aproblem that the team is facing. Where they are awareof the POEs, team members often do not have thereports from past POEs at hand and do not apparentlyuse POE results in daily decision-making.

Mid-level staff tend to be more aware of POEs. Inparticular, staff responsible for developing guidelinesand standards are often aware of POE results. Forexample, in the U.S. Postal Service, the staff who main-tain guidelines also administer POEs; the POEs con-ducted by the Administrative Office of the U.S. Courtsare used directly by the Judicial Conference to test andupdate the U.S. Courts Design Guide.

We were not able to find situations where seniormanagement used POEs for strategic planning. POEshave the potential for supporting “double-loop learn-ing” (Argyris and Schon, 1978)—that is, not only toevaluate how to achieve existing goals better but alsoto reflect on whether goals themselves need to bereconsidered. However, we were not able to find caseswhere this actually occurred.


We were not able to find many compilations of POEfindings, although several organizations such as theU.S. Army Corps of Engineers, U.S. Postal Service,Administrative Office of the U.S. Courts, General Ser-vices Administration, and others have incorporatedPOEs into design guides. Disney and the U.S. Depart-ment of State have incorporated POE into databases ofinformation. These are discussed in more detail below.

It does appear that POEs are not used for their fullpotential for organizational learning. In particular, wewere not able to find many circumstances where POEwas part of an active culture of testing decisions, learn-ing from experience, and acting on that learning. Thereare two major reasons for this:

1. Learning is fragile and difficult, and many organi-zations have not created the appropriate condi-tions for learning. If learning is to be genuinely“organizational,” a large number of staff musthave the opportunity to participate and to reflecton the results in a way that enables them to incor-porate the results into their own practice. Poten-tial participants must see the value for themselves:there must be incentives for participating. Also,evaluation will sometimes reveal that buildingperformance does not reach a desired standard.This is, of course, the value of POE, but manyorganizations punish people when innovations donot work. In addition, many organizations simplydo not make information available in a format thatis clear and useful to decision-makers.

2. Many organizations have not created a body ofknowledge that is valuable in the sense that it pro-vides a coherent, integrated body of knowledgethat is helpful in everyday decision-making.Knowledge tends to be informal and individual.

WAYS TO CREATE THE APPROPRIATECONDITIONS FOR LEARNING THROUGH POE

Create Broad Opportunities for Participationand Reflection

Our research suggests that POE-based knowledge isnot widely shared within most organizations. One wayto achieve this sharing is through direct participation inevaluations. Seeing how a facility works while hearingdirectly from users is a memorable experience. Also,the process of analyzing and writing up the results from

an evaluation can help decision-makers reflect on theimplications of the results and make links to their ownpractice.

A group of evaluators in New Zealand developed a“touring interview” methodology to allow decision-makers to actively participate in evaluations with littletraining and only modest commitment of time(Kernohan et al., 1992; Watson, 1996; 1997). Forexample, in an active evaluation program with morethan 80 completed evaluations, Bill Watson, a consult-ant to public and private clients, takes building userson a tour of the building and asks open-ended ques-tions—for example, “What works here?”—as well asmore specific probes about the functions of spaces andsystems. POE reports are mostly verbatim commentsby users and are sorted into categories such as “actionfor this building” or “change in guidelines for futurebuildings.” This approach is quite inexpensive and canbe completed with several person-days of effort. Theexperience is vivid for the participants and producesresults that are imageable and articulate. It also allowsparticipants to discuss relative priorities and values.However, because each touring interview group varies,it is more difficult to compare evaluations of differentsettings.

This kind of participatory evaluation can be anextension of existing processes for receiving feedbackfrom customers. Project managers in Santa ClaraCounty, California, were tired of receiving a storm ofrequests from users after they moved into a building.These were difficult to direct to contractors, suppliers,and others. They contracted with consultants CherylFuller and Craig Zimring to create a Quick ResponseSurvey (QRS) aimed at organizing and prioritizing userneeds about three months after buildings were occu-pied. All building users fill out a one-page question-naire, and project managers follow up with a half-daywalk-through interview of the building with the facilitymanager and staff representatives. The project man-agers then prioritize requests and meet with the clientorganizations. The State of California Department ofGeneral Services is further developing the QRS andwill have evaluators enter results into a lessons-learneddatabase.

A lessons-learned program initiated in 1997 for NewYork City to examine the success of school projects inthe state was aimed at participation by consultants. TheSchool Construction Authority (SCA), whose member-ship is appointed by the governor, the mayor, and theNew York City Board of Education, was charged with


the program. To get the program approved, SCA, underthe leadership of consultant Ralph Steinglass, adopteda simple methodology—require the architect or engi-neer of record to conduct the POE. The rationale wasthat this would guarantee that designers would confronthow users responded to their designs and force alessons-learned loop in the design process. About20 POEs have been completed. To ensure reliability,SCA reviewed the results before approving the POEs.In some cases, the architects or engineers had toreschedule their interviews when they were suspectedof introducing a bias or continue their investigation ifthey failed to include critical areas required in thestudy.

The three programs described above involve evalu-ation by the people who designed and managed theproject. As such, participatory evaluation is well suitedto supporting learning by in-house project managersand consultants. Whereas the New Zealand projects areled by consultants, the quick-response projects and theSCA projects are conducted entirely by the consultantsor project managers.

Create Incentives for Participation

Most building professionals are interested in doing agood job and see value in POE. However, as personaltime management consultant Stephen Covey hasargued, things that are merely important often lose outto things that are urgent: general benefits such as long-term learning often lose out when professionals arefaced with the pressing matters of everyday life. Whenmore specific incentives are offered, it often increasesparticipation in a POE program.

The drug company Ciba-Geigy has used direct mon-etary incentives. The architectural and engineering firmHLW and the contractor Sordoni Skansa Constructionput their design and construction profits ($300,000 and$1.2 million, respectively) at risk based on performanceon schedule, cost, and user satisfaction for the new $39million Ciba-Geigy Corporation’s Martin Dexter Labo-ratory in Tarrytown, New York. One-third of the profitswas based on user satisfaction responses to 14 surveyquestions: heating, ventilation, air conditioning, acous-tics, odor control, vibration, lighting, fume-hood per-formance, quality of construction (finishes), buildingappearance, and user-friendliness. The questions werebinary-choice (acceptable-not acceptable), and thebuilding had to reach 70 percent satisfaction to pass thetest. Some aspects such as sound transmission were

also assessed using physical measures; if the usersatisfaction measures did not reach criterion, physicalmeasures could be substituted (Gregerson, 1997). Thedesigners and contractors consulted the scientiststhroughout the process, showing them alternatives forthe façade design and full-scale mockups of the rangehoods. The building passed on all criteria except satis-faction with the range hoods, which were modifiedafter the evaluation as a response to user input. Whereassome aspects of this testing process might be question-able—Should you evaluate an entire building on 14yes-no scales? Should maintenance and operatingexperience be included?—this process gained from par-ticipation throughout. The design firm, contractors,management, and scientists all participated in estab-lishing the criteria at the outset, and the financialincentive encouraged the contractor and designers toconsult the users at every step in the process. It is diffi-cult to document the learning benefit of this process,but the contractor, Sordoni Skansa, has since usedPOE-based incentives in several other projects and hasrefined the way in which buildings are delivered.

The California Department of General Services isplanning to include the results of POEs as part of thereview of qualifications when selecting consultants andcontractors. This has strongly increased the interest inPOEs by participating firms. We are unaware of anyPOE programs that provide incentives for internal staffmembers to participate in evaluations, though severalprograms have discussed such incentives, such as pro-viding a free vacation day as a reward for adding datato the knowledge base or providing a mini-sabbaticalfor participating in evaluations or a lessons-learnedprogram. Disney provides a powerful, if indirect,incentive: knowledge. Only the industrial engineershave access to key data and they then become valuablemembers of the design team.

Reduce Disincentives: Create ProtectedOpportunities for Innovation and Evaluation

Organizational learning consultants have longpointed to an inherent contradiction in many organiza-tions. Whereas most organizations espouse innovationand learning, they behave in ways that limit it. Werecently participated in a meeting where an organiza-tion had used an innovative building delivery strategywith which it was not familiar. They had left out a keyreview step. When this became clear, a senior managerturned to the project manager and said: “We would


have expected someone at your level to do better.” Themessage to everyone in the room was clear: avoidinnovation and avoid evaluation! This syndrome—focusing on the individual rather than the performance,blaming the innovator rather than learning from theinnovation—is pervasive among organizations moregenerally (Argyris, 1992b; Argyris and Schon, 1978).However, some building-delivery organizations haveused POE to at least partially overcome it.

Some organizations have done this by explicitlysanctioning “research” with the attendant acknowledg-ment that innovations might not succeed. For example,the General Services Administration’s (GSA) PublicBuildings Service has recently appointed a director ofresearch. The first director, Kevin Kampschroer, has abudget to conduct, synthesize, and distribute research,including POE. The use of the term “research” carrieswith it the understanding that not all efforts are suc-cessful, and the budget provides some time for reflec-tion about findings. To date, much of the research isconducted by academic consultants who bring outsidelearning into GSA. However, GSA is also looking atways to broaden internal ownership of the researchprogram.

GSA has also created an active “officing” labora-tory in its own headquarters’ building. The lab, super-vised by Kampschroer, is one floor of actual workspacethat includes an innovative raised floor heating, venti-lating, and air-conditioning system and several brandsof modular office furniture systems. It also exploresdesign to support teamwork, with many small confer-ence rooms and meeting areas. The workers arefrequently surveyed and observed, and the lab alsobecomes a place where clients can see alternative officelayouts.

The U.S. Courts and the General Services Adminis-tration Courthouse Management Group are consider-ing developing a different kind of laboratory: a full-scale courtroom mockup facility where new courtroomlayouts and technologies can be tested and refined atrelatively low cost and risk. This facility, to be con-structed at the Georgia Institute of Technology, wouldallow mock trials to be conducted and would providetraining for judges, staff, and lawyers.

Another way to reduce the personal and organiza-tional cost of experimentation is by starting small withprojects that have an experimental component. Theinnovation can be evaluated and considered for broaderadoption. For example the U.S. Department of StateOffice of Overseas Buildings Operations (OBO) tries

out innovations on a limited number of projects beforerolling out the innovation to the larger organization.This office has recently used building serviceabilitytools and methods (Davis and Szigeti, 1996) for pro-gramming and design review for the new embassies inDar es Salaam and Nairobi.

The State of Minnesota Department of NaturalResources has used POE to evaluate two innovativeregional centers. In each of these cases the organiza-tional learning effort provided some additionalresources for data collection and reflection as well asthe clear designation that this was an innovative effortthat might not be fully successful.

In many organizations it is risky to be the first one totry an innovation. Massachusetts Institute of Technol-ogy organizational consultant Edgar Schein has pro-posed that while organizations may benefit greatlyfrom consultants, they often find the experience ofpeers more helpful when they actually move to imple-menting an innovation. Schein has called for “learningconsortia” where people can get advice from peers inother organizations and learn from their experience(Schein, 1995). He argues that although such learningconsortia may be effective at all levels of an organiza-tion, they are particularly effective among chief execu-tive officers (CEOs) or upper to midlevel managers.Although the prototype, laboratory, and learningconsortium efforts are quite different, all reduce thedisincentives for innovation and evaluation by allow-ing innovation and evaluation at relatively low personaland organizational cost.

Provide Access to Knowledge for Different Audiences

The simplest barrier to using POE for organizationallearning is when POE results are not available todecision-makers. Many organizations produce POEs ascase study reports that are not widely distributed. Partof this may be due to the history of POE, which hasfocused on single case studies, and part may be becauseof the perceived disincentives to distributing informa-tion that might be seen as critical of internal efforts orindividuals. Part of the problem is the simple technicaldifficulty of distributing printed information, and thishas become a lot easier with the Internet and intranetand virtual private networks. The National Aeronauticsand Space Administration makes its lessons learneddatabase available to all authorized staff and contrac-tors. In the United Kingdom, Adrian Leaman and BillBordass have created an interactive Web site for the 18


buildings they have evaluated as part of the PROBEproject. Funded by the Building Research Estab-lishment and Building Services Journal, PROBE standsfor post-occupancy review of buildings and their engi-neering.

Some organizations have overcome some of theseissues by creating design guides and databases of POEinformation. Agencies such as the AdministrativeOffice of the U.S. Courts, the U.S. Postal Service, andthe General Services Administration have createddesign guides that are widely distributed.

As we have suggested, the problem with organiza-tional learning is only partially technical. The tools forcreating Web sites and databases are now widely avail-able and inexpensive. A useful Web site requires theinitiative to collect the information, the time to makesense of it, and the will to share it.

Part of the problem with building delivery organiza-tions and design projects is that they represent manydifferent professional cultures. Engineers tend to take atechnical problem-solving approach. Architects areoften interested in form. Clients might be interested inthe usability and experience of the building. Seniormanagers might be searching for help in setting strate-gic directions, whereas project managers might beinterested in lessons learned about specific materials orequipment. Part of the challenge in creating any data-base or report is translating between these different pro-fessional cultures, and evaluators have not always beensuccessful at doing this.

Reduce Uncertainty by Upper Management’sCommitment

Participants in POE programs report that uncertaintyabout senior management’s commitment to the pro-gram is a key disincentive to participation. Sometimesthe lack of commitment is seen in lack of resources, butit is often manifest in lack of visible endorsement forthe program and lack of commitment to the two- tofive-year time span necessary to see results in terms oforganizational learning. Savvy staff have learned notto genuinely commit to the management’s infatuationdu jour, knowing that it will change quickly.

CREATING A KNOWLEDGE BASE FOR BUILDINGDELIVERY AND MANAGEMENT

Most fundamentally, organizational learning for abuilding delivery organization is producing better

buildings more effectively. Given the large number ofPOEs that have been completed—longtime researcherand Environment and Behavior Journal editor RobertBechtel estimates that more than 50,000 have beencompleted—one would expect that there would havebeen many books and guides that synthesize the resultsof POEs and tie them into a coherent guide for keyprogramming and design decisions. However, suchguides are relatively rare. In part, this is because of thefocus of POE researchers and consultants on casestudies. Much knowledge about buildings has beenbuilt up incrementally through negotiation on indi-vidual projects and programs, but organizations seldomtake the time to identify the key strategic decisions thatmost affect their clients. Efforts such as AmericanSociety for Testing and Materials (ASTM) BuildingServiceability Tools and Methods have begun to do thisfor programming and portfolio management, but wehave seldom done this for POE.

In this section we examine several strategies thathave proven successful for beginning to create this kindof knowledge base. In several cases, organizations havebuilt on POEs that have been initiated for different pur-poses (or seem to us to be able to do so reasonablyeasily).

POE can be particularly successful in organizationallearning if it links strategic facilities decisions to the“key business drivers” of the client organization. In the1970s, the U.S. Army was shifting to an all-volunteerarmy. Potential recruits said that the aging facilitieswere a significant impediment to recruiting and reten-tion, and the Army sought to renovate or rebuild manyof its buildings. To help guide the multibillion dollarinvestment, the Army Corps of Engineers created alarge program of participatory programming and evalu-ation, resulting in some 19 design guides (Shibley,1985).

In the 1980s, the newly reorganized U.S. Postal Ser-vice (USPS) was losing customers to private competi-tors such as FedEx and UPS (Kantrowitz and Farbstein,1996). Focusing initially on the customer experiencewith lobbies, the USPS contracted with Min Kantrowitzand Jay Farbstein and Associates to conduct focusgroup evaluations. This has led to a large and con-tinuing program of evaluations and design guidedevelopment. New concepts of post office design aredeveloped such as the retail-focused “postal store,”innovative projects are designed, the projects are evalu-ated, and the ideas are refined and then incorporatedinto design guides. This program has sustained an


ongoing process of testing and refining the designguides through evaluation and experience. Morerecently, the USPS has de-emphasized on-site evalua-tions. Most POEs now involve having facility managersfill out relatively brief mail-out surveys. The POE man-ager has found that the open-ended responses to thequestionnaire are often most valuable in refining theUSPS design guidelines because they are more spe-cific than the scaled satisfaction responses.

BUILDING ON EXISTING EVALUATIONS

An organization can begin to rationalize its knowl-edge base by building on evaluations that occur forother reasons. As the experienced evaluator BobShibley has said, evaluations are easiest to justify ifthey bring project support, analysis benefits, andlessons-learned benefits (Shibley, 1985).

Building on Diagnoses of Troubled Settings

Sometimes a building is the subject of complaints orcontroversy; a POE can help to diagnose the source ofproblems and prioritize solutions. For example, the newSan Francisco central library was a landmark when itopened in 1996, but it faced immediate controversy.Some of the initial programmatic assumptions con-tinued to be debated—such as the wisdom of movingbooks to closed stacks to create room for computers.As a result, the mayor appointed an audit commissionthat recommended a POE, led by architect CynthiaRipley and including the director of the Los Angeleslibrary system. After interviewing staff and users,observing use, and analyzing records, the POE teamhighlighted problems with way-finding, flexibility, andpublic access to books. The POE recommendeddetailed renovations to reorganize the stacks and col-lection (Flagg, 1999; Ripley Architects, 2000). ThePOE is quite thorough in suggesting detailed specificchanges, and the basis of these recommendations couldpotentially be turned into planning principles. Most sig-nificantly, this raises issues of programming processwhere the (former) library director went against therecommendations of his planning committee to reduceaccess to books in favor of closed stacks. It can supportbroader reflection about the role of libraries and physi-cal structures in providing information in the age ofcomputers.

Whereas the focus on understanding problems andfailure provides a clear direction to POEs and has a

history in case studies of blast, earthquakes, and otherbuilding failures, these kinds of POEs carry specialrisks of becoming ways to focus (or deflect) blame.

Capitalizing on Evaluations of Innovations

Evaluation can help decide whether innovativebuildings or building components should be consideredfor additional capital investment. For example, as men-tioned above, the State of Minnesota Department ofNatural Resources (DNR) has recently changed theway in which it manages the environment. Rather thanorganizing its staff by discipline, DNR now uses amatrix management system where decisions are madeby a multidisciplinary group organized by ecosystem.The DNR is creating new regional centers that includewildlife biologists, air and water specialists, and othersconcerned with a given area. The centers are intendedto encourage multidisciplinary collaboration and to bevery “green.” The DNR contracted with a universityteam led by Julia Robinson to evaluate two of the ini-tial projects. The team made numerous recommenda-tions. The POE was included as an appendix for thefunding request for the third center. This was the firsttime in DNR’s history that a capital request was fullyfunded on the first attempt, and the DNR was told thatthe POE was a major reason: it showed a high level ofunderstanding of the project. This result provided animportant incentive for DNR as an organization. How-ever, the project also raised some issues aboutsustainability, and the internal staff did not feel thatthey had been fully consulted in the POE process. Anadditional team was hired to create design guidelinesin close consultation with staff (M. Wallace, personalcommunication, 2000). Issues such as sustainability,which are undergoing rapid change, are particular can-didates for “double-loop” learning where both goalsand methods for achieving them are being developed,if appropriate conditions are established for discussion,reflection, and action.

Focus on “Learning Moments”

The Administrative Office of the U.S. Courts (AO)conducts a POE program that informs guidelines (inthe U.S. Courts design guide). However, the AO hasachieved organizational learning by linking the designguide to a strategic learning moment in the develop-ment of courthouses: the negotiation between judgesand the building agent (the General Services Adminis-


tration) about the scope and quality level for new court-houses. In the early 1990s, the U.S. governmentinitiated the largest civilian construction program sincethe Second World War, projecting to spend more than$10 billion on 160 new courthouses. (The creation ofnew judgeships in the 1980s, concerns for increasedsecurity, and new technologies all necessitated newcourthouses or major renovations.) However, both thejudiciary and the GSA were being criticized by Con-gress for creating marble-clad “Taj Mahals.” The AOinitiated the POE program to identify necessarychanges to the standards in the first edition of the designguide, to defend the judiciary against attack by docu-menting the efficacy of the design standards, and toinform the negotiation about issues such as the dimen-sions and materials of courtrooms and chambers.Information from POEs was also used in training work-shops for judges and staff who were becoming involvedin new courthouse design and construction. This pro-gram is run by the AO, but the design guide is actuallycreated and vetted by a committee of the Judicial Con-ference, the group that sets broad policy within the fed-eral judiciary. This program is quite unusual: it is theonly case that we are aware of where a POE and designguide are developed by a client organization that doesnot build its own buildings.

The Minnesota Department of Natural Resourcesproject also focuses on strategic moments, especiallythe approval of the funding package by the key legisla-tive committees.

The focus on strategic learning moments is similarto Shibley’s reminder that information is most likely tobe used when it is asked for (Shibley, 1985). A strate-gic learning moment is a critical time when informa-tion or a POE can help resolve a problem or issue thatis of considerable importance to the participants. Thefocus on learning moments can also be used in devel-oping policy documents or targeting POEs towarddecisions.

Creating Organizational Memory for Precedents

A key part of organizational memory is simplyknowing what the organization has done, but few POEprograms have been linked to recording and analysis ofprecedent. There is a real opportunity to link evalua-tion to a record of past projects. This record can includesimple plans and photos and some analyses of cost,size, and materials. These descriptions can be linked toevaluations.

LESSONS FROM POE PROGRAMS: ENHANCINGORGANIZATIONAL LEARNING

We have suggested that POE has a large potentialfor lessons learned as well as for project support andanalysis. Because of the historic focus of much POEresearch, the difficulty of finding resources for organi-zational learning, and sensitivities in exposing prob-lems, relatively few organizations have created effec-tive POE-enabled organizational learning systems thatinclude:

1. monitoring changes in the internal and externalbusiness environment,

2. establishing performance goals based on internaland external influences,

3. assessing performance,4. interpreting and discussing the implications of

results,5. consolidating results into an organizational

memory,6. widely distributing findings and conclusions,7. creating a culture that allows the organization to

take action on the results,8. taking action based on organizational learning.

However, based on a number of successful exam-ples, we suggest the following strategies for creatingthe conditions for learning:

• Create opportunities for participation• Add incentives• Remove disincentives• Provide access to information• Provide upper-level management support

Successful organizations have also used severalstrategies for creating knowledge:

• Clarify key strategic choices• Build on existing evaluations• Focus on strategic moments• Record precedents

The lessons from these 18 POE programs, whichinfluence billions of dollars of construction, suggestthat the solution to creating a lessons-learned programis partly technical, such as using information tech-nology to reduce costs in gathering and distributinginformation. At its core, however, the problem is orga-


nizational—creating a setting in which decisions canbe evaluated, discussed, and learned from.

ABOUT THE AUTHORS

Craig Zimring is a professor of architecture and ofpsychology at the Georgia Institute of Technology. Inhis teaching, writing, consulting, and research he hasdeveloped methods, procedures, and concepts for theevaluation of buildings, including comprehensivestudies of building types such as healthcare facilities,jails and prisons, courthouses, and specialized studiesof wayfinding, security, stress, and other issues.Dr. Zimring has focused on how social, organizational,and behavioral information can be incorporated intodesign and decision making at a variety of scales, froma freshman design studio to the $4.5 billion Californiaprison development program, the $6 billion FrenchUniversities 2000 program, and the $1 billion annualconstruction budget of the California Department ofGeneral Services. He has worked in the design studio,lectured to facility managers, written in the popular andprofessional press, served as a consultant and directedresearch projects for AT&T, U.S. Department of State,U.S. General Services Administration, the Administra-tive Office of the U.S. Courts, U.S. Department ofTransportation, Ministry of Education of France, andmany others and served on the boards of several pro-fessional organizations including the EnvironmentalDesign Research Association and the Justice FacilitiesResearch Program. Dr. Zimring was a distinguishedsenior visiting fellow at the Centre Scientific et Tech-nique du Batiment, Paris; he has received awards fromthe American Society of Interior Designers and theNational Endowment for the Arts Design ResearchRecognition Program. He holds a bachelor of sciencefrom the University of Michigan, and a master of sci-ence and a Ph.D. from the University of Massachusettsat Amherst.

Thierry Rosenheck is a project manager at the U.S.Department of State, Office of Overseas BuildingsOperations (OBO), and has been working on embassyrehabilitation projects in New Delhi, Beirut, Tel Aviv,and Jerusalem since 1999. He has developed a service-ability profile of user generic requirements for newchancery office buildings using the ASTM Standard onWhole Building Functionality and Serviceability andwith the Centre for International Facilities. He hascoordinated POE and serviceability input with other

ongoing projects at the Office of Overseas BuildingsOperations. Prior to working with the Department ofState, Mr. Rosenheck was in private practice. He hasworked for architectural firms, a construction firm, andtaught at the School of Architecture and Urban Plan-ning at Howard University. He holds a bachelor ofarchitecture degree from Howard University, amaster’s degree in architecture and environment-behavior studies from the University of Wisconsin-Milwaukee, and is a licensed architect in the District ofColumbia.

REFERENCESArgyris, C. (1992a). On Organizational Learning. Cambridge, Mass:

Blackwell.Argyris, C. (1992b). Teaching smart people how to learn. In: C. Argyris

(Ed.) On Organizational Learning (pp. 84-100). Cambridge, Mass:Blackwell Business.

Argyris, C., and Schon, D. (1978). Organizational Learning. Reading,Mass: Addison-Wesley.

Baird, G., Gray, J., Isaacs, N., Kernohan, D., & McIndoe, G. (Eds.). (1996).Building Evaluation Techniques. New York: McGraw-Hill.

Bechtel, R. (2000). Personal Communication.Bordass, W., and Leaman, A. (1997). Future buildings and their services:

strategic considerations for designers and clients. Building Researchand Information 25(4): 190-195.

Campbell, D.T. (1999). Social Experimentation. Thousand Oaks, Califor-nia: Sage Publications Inc.

Cohen, R., Bordass, W., and Leaman, A. (1996). Probe: A Method of Inves-tigation. Harrogate, United Kingdom: CIBSE/ASHRAE Joint NationalConference.

Davis, G., and Szigeti, F. (1996). Serviceability tools and methods (STM):Matching occupant requirements and facilities. In: G. Baird, J. Gray, N.Isaacs, D. Kernohan, G. McIndoe (Eds.) Building Evaluation Tech-niques. New York: McGraw-Hill.

Flagg, G. (1999). Study finds major flaws in San Francisco main library.American Libraries 30(9):16.

Friedmann, A., Zimring, C., and Zube, E. (1978). Environmental DesignEvaluation. New York: Plenum Press.

Gregerson, J. (1997). Fee not-so-simple. Building Design and Construction(August): 30-32.

Horgen, T.H., Joroff, M.L., Porter, W.L., and Schon, D.A. (1999). Excel-lence by Design: Transforming Workplace and Work Practice. NewYork: Wiley.

Huber, G.P. (1991). Organizational learning: The contributing processesand the literature. Organization Science 2: 88-115.

Kantrowitz, M., and Farbstein, J. (1996). POE delivers for the Post Office.In G. Baird, J. Gray, N. Isaacs, D. Kernohan, G. McIndoe (Eds.) Build-ing Evaluation Techniques. New York: McGraw-Hill.

Kernohan, D., Gray, J., and Daish, J. (1992). User Participation in BuildingDesign and Management: A Generic Approach to Building Evaluation.Oxford: Butterworth Architecture.

Leaman, A., Cohen, R. and Jackman, P. (1995). Ventilation of office build-ings: Deciding the most appropriate system. Heating and Air Condi-tioning (7/8): 16-18, 20, 22-24, 26-28.

McLaughlin, H. (1997). Post-occupancy evaluations: “They show us whatworks and what doesn’t.” Architectural Record 14.

Preiser, W. F. E. (1994). Built environment evaluation: Conceptual basis,benefits and uses. Journal of Architectural and Planning Research11(2), 92-107.


Preiser, W.F.E., Rabinowitz, H.Z., and White, E.T. (1988). Post-OccupancyEvaluation. New York: Van Nostrand Reinhold.

Preiser, W.F.E., and Schramm, U. (1997). Building performance evaluation.In D.Watson et al. (Eds.) Time-Saver Standards (7 ed., pp. 233-238).New York: McGraw-Hill.

Raw, G. (1995). A Questionnaire for Studies of Sick Building Syndrome.BRE Report. London: Construction Research Communications.

Raw, G. (2001). Assessing occupant reaction to indoor air quality. In: J.Spengler, J. Samet, and J. McCarthey (Eds.) Indoor Air Quality Hand-book. New York: McGraw-Hill.

Ripley Architects. (2000). San Francisco Public Library Post OccupancyEvaluation Final Report. San Francisco: Ripley Architects.

Schein, E.H. (1995). Learning Consortia: How to Create Parallel LearningSystems for Organization Sets (working paper). Cambridge, Mass:Society for Organizational Learning.

Schneekloth, L.H., and Shibley, R.G. (1995). Placemaking: The Art andPractice of Building Communities. New York: Wiley.

Shibley, R. (1982). Building evaluations services. Progressive Architecture63(12): 64-67.

Shibley, R. (1985). Building evaluation in the main stream. Environmentand Behaviour 1985(1):7-24.

Vischer, J. (1996). Workspace Strategies: Environment as a Tool for Work.New York: Chapman and Hall.

Watson, C. (1996). Evolving design for changing values and ways of life.Paper presented at the IAPS14, Stockholm.

Watson, C. (1997). Post occupancy evaluation of buildings and equipmentfor use in education. Journal of the Programme On Educational Build-ing (October).

Zeisel, J. (1975). Sociology and Architectural Design. New York: RussellSage Foundation.

Zimring, C.M., and Reizenstein, J.E. (1981). A primer on post-occupancyevaluation. Architecture (AIA Journal) 70(13): 52-59.

Zimring, C.M., and Welch, P. (1988). Learning from 20-20 Hindsight.Progressive Architecture (July), 55-62.

54

6

The Role of Technology for BuildingPerformance Assessments

Audrey Kaplan, Workplace Diagnostics Ltd.

To date, building performance assessments havebeen limited, for the most part, to large organizationsand institutions. The substantial investment of time andmoney needed to mount such assessments using tradi-tional methods is a major hurdle for most midsize andsmall organizations or for niche groups. Relative to thenumber of facilities and organizations that couldbenefit from such assessments, too few are done tomake them an effective tool for aligning occupant orstakeholder expectations with the finished product.Advances in electronic communications and easy-to-use Web browsers now make it attractive to conductthese assessments via the Internet or an intranet.“Cybersurveys” or “e-surveys” (polls or assessmentsadministered electronically) represent the breakthroughin social science research that will make this workcheaper and more effective to complete. Moreover, theinnovative medium will likely inspire the invention ofwholly new work and objectives that could not be doneusing existing assessment methods (Bainbridge, 1999).

Regardless of the medium, all assessment workbegins with good research design, clearly stated objec-tives, solid planning, and preparation to conduct a sur-vey, complete the analysis, and produce reports. Thischapter addresses issues related to implementingcybersurveys and assumes that sound principles ofbuilding performance assessment are in place. Infor-mation technology has unique features to gather feed-back about building performance, which in turn can beused to improve facility management and acquisition.

As the set of Internet users begins to reflect the popu-lation in general, or a specific group under study,cybersurveys may become the predominant method ofadministering building assessments. If widely used, the

assessment process can be an effective tool for con-tinually improving the value of each facility, regard-less of its size or unique occupancy.

INTRODUCTION

Cybersurveys complement existing survey methodsof assessment, such as paper, mail, phone, in-personinterview, site visits, expert observations, photography,and instrument measurements.

Instruments to assess buildings continue to improvesteadily. Prompted by the energy crisis of the 1970s,instruments were developed to measure and/or diag-nose the effectiveness of building systems (e.g., walls;windows; roofs; heating, ventilation, and air condition-ing; lighting). Generic methodologies were conceivedto diagnose total building systems, and tools weredesigned to assess the performance of specific buildingaspects. See, for example, A Generic Methodology forThermographic Diagnosis of Building Enclosures (Milland Kaplan, 1982) for a methodology and tools todetermine heat loss and uncontrolled air flow acrossbuilding enclosures. The objective of methodologiesand tools developed at that time was to better under-stand the cause of deficiencies—be they rooted indesign or construction—and then to prescribe correc-tive actions and processes to prevent future occur-rences.

In the 1980s, the heightened concern for the qualityof interior environments drove the development of newtools to better represent the total environmental settingthat occupants experience. Many traditional lab instru-ments were modified (or put on carts) and brought intooccupied buildings. The focus was to record environ-

THE ROLE OF TECHNOLOGY FOR BUILDING PERFORMANCE ASSESSMENTS 55

mental factors that influence human comfort and per-formance (e.g., light, sound, temperature, relativehumidity, air quality, spatial configuration, aesthetics).These instruments were often cumbersome or too deli-cate for such robust applications, so manufacturers andresearchers alike redesigned them to better match thefield conditions and the nature of the data being col-lected (see, for example, Schiller et al., 1988, 1989).Some of the features modified to make reliable fieldinstruments included narrowing the instruments’ range,adjusting the sample duration and frequency, placingsensors to record levels in occupied zones (as opposedto air supply ducts, etc.), and immediate statisticalanalysis of select field data for preliminary interpreta-tion to guide the balance of the data collection. Sincethen, there have been steady, incremental improve-ments to the set of field instruments, benefiting fromminiaturization and faster processing as the whole com-puter industry developed.

Academic centers continue to advance the use ofinstruments to assess building performance. See, forexample, the Center for Environmental DesignResearch (also known as the Center for the Built Envi-ronment) at the University of California, Berkeley,www.cbe.berkeley.edu, and Cornell University, Ithaca,New York, ergo.human.cornell.edu. Dependent uponfunding to support building performance programs,these and other institutes lend instruments and materialsto architecture, engineering, and facility managementschools. These lending programs are intended toencourage the next generation of built environmentprofessionals to build and manage environmentallyresponsible and energy-efficient structures. Forexample, the Vital Signs program at the University ofCalifornia, Berkeley (1996), assembled tools to mea-sure a range of building performance attributes. Thesewere packaged into kits that educators could borrowfor their classes. Support funding has since expired,however other programs carry on toward the sameobjectives. See, for example, the occupant surveyproject <www.cbe.berkeley.edu/cedr/cbe>.

Improvements continue in the measurement of thephysical built environment, but there are no majorbreakthroughs or fundamental changes in how or whatis measured. Similarly, important advances in opinionresearch, such as telephone interviews, did not funda-mentally change the way data were collected and ana-lyzed or questionnaires were designed (Taylor, 2000).However, the visual and responsive characteristics of

cybersurveys offer new and significant opportunitiesin the way building assessments are conceived andcompleted. The author sees cybersurveys as the nexttechnology-based advance in building performanceassessments. The remainder of this chapter discusseshow this technology is used to assess building perfor-mance.

CYBERSURVEYS

The unique features of computer technology (e.g.,Web, e-mail, electronic communications) improveupon and add efficiency to the polling process. Thetraditional methods of surveying people—telephone,mail, or in person—are expensive and tend to be usednow only by large organizations. The negligible cost todistribute surveys over the Internet or intranet and thepotential to reach far and wide are changing the surveyindustry. Now, small and medium-size businesses usecybersurveys as a means to gather valuable informa-tion from customers, potential customers, employees,or the general public. Readily available survey devel-opment products help the nonspecialist to build a Websurvey questionnaire, publish it on the Web, collect thedata, and then analyze and report the results (seeAppendix E). These tools address how to conduct theresearch and the mechanics of data collection. They donot, and cannot, replace the management decision toconduct the inquiry, how to interpret the numbers, orhow to get actionable results from the effort.

Cybersurveys are distributed as e-mail to a specificaddress (either embedded in the message or as anattachment) or openly on the Web. There are more con-trol and layout options with Web-based surveys thanwith e-mail. With HTML (hypertext markup language),attractive and inviting forms can be created. Some ofthe survey software packages have automated routinesto design the survey’s physical appearance.

The cost of initial data collection is high—whetherfor personnel to conduct interviews and code data orfor setting up a computer-based survey to automati-cally solicit and compile data. However, the biggestissues currently facing cybersurveys are the lowresponse rate (compared to traditional methods andrelative to the potential based on number of visits to asite) and ensuring a proper sample. These issues arediscussed in the next two sections, followed by lessonslearned from Web-based surveys.


RESPONSE RATE

It is widely agreed that the more attempts made tocontact respondents, the greater are the chances of theirreturning a survey. Similarly, a greater number ofresponse formats increases the likelihood of moreresponses. Multiple contacts offering alternate ways ofresponding with appropriate personalization, singly andin combination, increase response rate. For example,Schaefer and Dillman (1998) reported from publishedstudies the average response rate for e-mail surveys.With a single contact, the response rate was 28.5 per-cent, with two contacts it was 41 percent, and with threeor more contacts the response rate was 57 percent.Cybersurveys offer another contact vehicle and methodof response (e.g., interactive display, print and return)to the options already available. In and of itself, use ofthis vehicle contributes to higher response rates.

A consistent finding is that cybersurveys are returnedmore quickly than mail responses. In comparing sev-eral studies that used both e-surveys and paper (mailand fax), response speeds ranged from 5 to 10 days forelectronic distribution versus 10-15 days for mail(Sheehan and McMillan, 1999).

To date, response rates from e-surveys have beenlower than for postal mail. E-mail rates range from 6 to75 percent. However the electronic and mail surveysused for this comparison (Sheehan and McMillan,1999) had small sample sizes (less than 200) and variedwidely in survey topic and participants’ characteristics,so the spread is not surprising. E-surveys often achievea 20 percent response, which is half of the rate usuallyobtained by mail or phone surveys. Some e-surveyshave had less than 1 percent return of the possibleresponses, based on the number of hits to a Web site(Basi, 1999). There were some exceptions to this trendin the early 1990s when e-surveys were distributed toworkers in electronic-related businesses (e.g., tele-phone and high-tech sectors). At that time, there wasstill a high-tech allure and/or novelty to receivinge-mail, so response rates were acceptable (greater than60 percent). This highlights how quickly the field ischanging. Lessons learned from early cybersurveysmay not necessarily apply in a current context. SeeAppendix E for a discussion of the changing context ofon-line communications.

Other reasons for low participation in on-line sur-veys may involve a reluctance to share one’s views in anontraditional environment. However, for those whoare comfortable with this arrangement, replies to open-

ended questions are richer, longer, and more revealingthan those from other methods (Taylor, 2000). Cyber-surveys are nontangible. Paper surveys may sit onsomeone’s desk and serve to remind him or her tocomplete them. Individuals intending to complete acybersurvey may bookmark the survey’s URL (uni-form resource locator) for later use but then forget toretrieve it.

SAMPLING

The Web is worldwide. People anywhere on theplanet can answer a survey posted on the Web even ifthey are not part of the intended audience (unless con-trols are used, such as a unique URL, a password, orsetting the meta tags so that the page is not picked upby search engines). To date, cybersurveys are com-pleted by accidental volunteers (they stumbled uponthe survey) and self-selected individuals who chooseto participate.

Because there is no central registry of Web users,one does not know who is on-line, so one cannotreliably reach a target population (see Appendix E fordetails of who is on-line and where they are geographi-cally). Moreover, there is no control on how electronicmessages are cascaded or linked to other sites. A mes-sage can be directed to select individuals or groupswho, for their own reasons, send it along to others. Thisbehavior makes it difficult to ensure that the surveyreaches a random or representative sample of respon-dents. Despite these sampling difficulties, careful useof nonprobability sampling can produce results thatrepresent a specific subset of the population (Babbie,1990).

LESSONS LEARNED

The following recommendations for Web surveysare adapted from published papers, Web sites, and theauthor’s experience (Bradley, 1999; Kaye and Johnson,1999; Sheehan and McMillan, 1999; Perseus, 2000).Recommendations are organized under the topics ofWeb survey design considerations, sampling, publicity,and data collection and responses.

Web Survey Design Considerations

1. Keep the survey as short as possible for quickcompletion and minimum scrolling.


2. Use simple designs with only necessary graphicsto save on download time.

3. Use drop-down boxes to save space, reduceclutter, and avoid repeating responses.

4. State instructions clearly.5. Give a cutoff date for responses.6. Personalize the survey (e.g., use recipient’s e-mail

address in a pre-notification letter; identify thesurvey’s sponsor, which can also add credibility).

7. Assure respondents that their privacy will be pro-tected—both their e-mail address and the safetyof their computer system (virus free).

8. Conduct pre-tests to measure time and ease forcompleting the survey. Electronic pre-testing iseasier and cheaper than traditional methods.

9. Try to complete the survey using different browsersto uncover browser-based design flaws.

10. English-language cyber-surveys can easily over-come geographic barriers (Swoboda et al., 1997).See Table 6-1 for the percentage of Internet userswho prefer to use the Web in English rather thantheir native language.

Sampling

1. To generalize the results to a wider population,define samples as subsets of Web users based onsome specific characteristic.

2. Solicit responses from the target population. Thiscan be done by linking the survey from selectsites, by posting announcements on discussion-type sites that the target population is likely touse, or by selecting e-mail addresses posted onkey Usenet newsgroups, listserves, and chatforums. Invited participants should be given anidentification number and password, that are bothrequired to access the Web-based survey.

3. Clearly state the intended audience in the intro-duction to the survey so, hopefully, only those towhom it applies will respond.

Publicity

1. Systematically publicize the survey daily throughvarious means. Create awareness of the survey ata wide variety of Internet or intranet outlets toreduce bias. “Pop-up” surveys may attract moreresponses than simple banner invitations that arefixed to a page. Advertising on select sites mightincrease the number of completions but does notcontinually invite (spam) a few discussion groupswhile ignoring others. See Table 6-2 for methodsto invite people to a survey.

2. Pre-notify respondents who would find the surveyrelevant (e.g., those to whom an important issueis current or timely).

3. Do not go overboard on publicity, just announcethe survey.

4. List the survey with as many of the major searchengines as possible. Services, such as “Submit It,”send listings to many search engines with just oneentry. Then use different search strategies and termsto locate the survey and uncover glitches or errors.

TABLE 6-1 Internet Users, by Region, Who Preferto Use the Web in English Rather than Their NativeTongue.

Region % Who Prefer English

North America 74Africa, Middle East 72Eastern Europe 67Asia 62Western Europe 51Latin America 48

Source: USA Today (2001).

TABLE 6-2 Methods to Attract Respondents to a Cybersurvey.

Announcements Browsing intercept software Customer recordsE-mail directories Harvested addresses Hypertext linksInterest group members Invitations (banners, letters, etc.) Pop-up surveysPrinted directories Registration forms SnowballingStaff records Subscribers Web site directories

Source: Bradley (1999).


5. Check that the survey links remain posted and thatthe URL is visible on the page.

6. Write the complete URL in announcements ormessages because it can be an easy, “clickable”link in some e-mail transmissions.

7. Ask respondents how they found out about thesurvey to gauge which sites and discussion out-lets were most effective in reaching the targetaudience.

8. If appropriate and/or feasible, offer incentives forcompleting the survey (ranging from the surveyresults to lottery tickets, money, or discounts fora third-party product). Check the laws governingincentives because these vary by jurisdiction.

Data Collection and Responses

1. Design the survey so that it is submitted with asimple click of the mouse or keystroke.

2. Upon receiving the survey, set up for an auto-matic thank-you reply to the sender saying thatthe survey was successfully transmitted.

3. To check for duplicate responses, ask for respon-dents’ e-mail address and/or track senders’ Internetor intranet protocol address.

4. For ease of importing e-mailed data into a statis-tical software program, design the return formsso that each question is listed on one line followedby its response and a corresponding numericalvalue.

5. People may respond to electronically viewedscales differently than to scales that are spoken oron paper. Taylor (2000) observed that fewerpeople picked the extremes on e-scales than whenthey heard the questions and response options.

CONCLUSIONS AND DISCUSSION

The Internet or intranet and World Wide Web enabletraditional building performance assessments to beaccomplished more cheaply and effectively. The elec-tronic medium may well become the primary surveyvehicle owing to its convenience, low-cost of distribu-tion and return, ability to verify (check for errors) andreceive data—including rich text replies—in electronicformat, and the fact that it is an easy way to giverespondents feedback. All this is done incredibly fastwith an attractive, visual presentation.

The opportunities presented by the medium itselfwill likely lead to the invention of new products,

methods, and presentations for building performanceassessments. For example, new lines of research mightbe conceived that integrate occupants’ perception ofinterior environments with data from the building con-trol system and/or weather conditions or that track theuse of facilities as recorded by smart cards and/or thespatial coordinates of workspaces as recorded bycomputer-aided drafting systems.

Select opinion leaders and/or professionals could bedrawn to an Internet-based “brain trust” that focuses onspecial questions related to the built environment. TheInternet or intranet and Web are excellent media forestablishing virtual libraries and databases that manycan access and use. Establishing sites with existing dataallows other researchers to produce interpretations,conclusions, or knowledge that is in some way differ-ent from that produced in the original inquiry (i.e., sec-ondary analysis). This work would economically andefficiently add to the experience and knowledge ofbuilding performance assessments.

With cybersurveys, questions can be asked of any-one, worldwide, who is on-line. It is feasible to poll awide population about issues that impact a specificbuilding. The sample is not limited to those who workin, visit, or are somehow involved with a building. Tap-ping into these opinions becomes important when thereis a highly visible construction, a new and importantpublic structure, or a controversy connected with afacility. Public opinion might provide valuable insightto how a building, its occupants, or a particular issue isviewed. For example, an incidence of sick-buildingsyndrome or an environmental disaster might be officiallyresolved to those managing a facility, but the local com-munity could remain suspicious and unsupportive ofthe organization. Using wide-reaching cybersurveys totap into that sentiment gives insight that might influ-ence decisions regarding the building’s publicity, image,operational management, and the communication tousers and other interested parties of what was done.

There is great value in being able to reach a popula-tion that is not yet at a facility. For example, high schoolstudents are actively on-line and, through cybersurveys,can be asked their expectations for their universityexperience—how they will live, learn, study, and makefriends. It would be insightful to determine that theyhave, for example, no desire to study in traditionallibraries or in their dormitory rooms. They may expect“cybercafés,” collaborative work settings, or learningenvironments that ease their transition into work envi-ronments where they will earn a livelihood. Through


select demographic questions, survey researchers cancharacterize the future university population and sug-gest built environment features compatible with theirexpectations. Universities may use these as features toattract top students and then to elicit the best work fromenrolled students.

Much of the young, skilled work force in the tech-nology sector comes from outside North America andfrom different cultures. Each has its own expectationof work and the workplace. Here again, using cyber-surveys to understand their expectations of the work-place and to educate them about North American work-places can speed their integration into the work force.There is a strong business case to support this use ofcybersurveys since they readily translate into businessobjectives of worker productivity and reduced time tomarket for products.

Occupant feedback is difficult to get, or unreliable,when people are stressed, such as when they need hos-pitals, senior residences, and funeral parlors. However,after the crisis has passed and their emotions are stable,they could offer insight to aspects of the physical set-ting that helped them at the time or that would havebeen of benefit. Cybersurveys can reach these peopleat a time when they can offer measured opinions on thebuilt environment.

The rewards of cybersurveys are so rich that theirpotential will surely be used and the methodologicaland sampling issues of today will be resolved. It is hardto imagine future building performance assessmentswithout extensive use of the Internet or intranet and theWeb. However, traditional methods will continue to beused, perhaps becoming a smaller part of the overallwork. Printing did not fully replace handwriting, radiodid not take the place of newspapers, and television didnot supplant movies or radio. Cybersurveys enablemany things that could not be done or afforded usingtraditional methods. The addition of this medium to thetools for building performance assessment greatlyenhances the accessibility and value of such data andof the evaluation process itself.

ABOUT THE AUTHOR

Audrey Kaplan is president of Workplace Diagnos-tics, Ltd., an Ottawa-based consulting company thatspecializes in the evaluation and design of workspace.Ms. Kaplan has been actively involved in building per-formance as a research scientist and consultant for 20years and has published widely in the field. Her

research activities range from the evaluation of com-fort and environmental quality in offices, to the assess-ment of total building performance, to the design ofworkstations with personal environmental controls. Inaddition to consulting, Ms. Kaplan is a regular speakerat professional conferences and continuing educationseminars, and is a certified instructor for professionaltraining courses with the International Facility Man-agement Association (IFMA). She was an adjunct pro-fessor at the University of Manitoba and served onIFMA’s international board as a director from 1998-2000, as chair of IFMA’s Canadian Foundation; and asa trustee of IFMA’s foundation. Ms. Kaplan receivedIFMA’s 1996 Distinguished Author Award for her co-authored book Total Workplace Performance: Rethink-ing the Office Environment. She holds a bachelor ofarchitecture from Carleton University and a master ofscience in architecture from Carnegie MellonUniversity.

REFERENCESBabbie, E. (1990). Survey Research Methods. Belmont, California:

Wadsworth.Bainbridge, W. (1999). Cyberspace: Sociology’s natural domain. Contem-

porary Sociology 28(6):664-667.Basi, R. (1999). WWW response rates to socio-demographic items. Journal

of the Market Research Society 41(4):397-401.Bradley, N. (1999). Sampling for Internet surveys: An examination of

respondent selection for Internet research. Journal of the MarketResearch Society 41(4):387-395.

Center for Environmental Design Research (known as the Center for theBuilt Environment) (1996). Vital Signs. Berkeley: University of Cali-fornia (www.cbe.berkeley.edu).

Kaye, B., and Johnson, T. (1999). Research methodology: Taming the cyberfrontier. Social Science Computer Review 17(3):323-337.

Mill, P., and Kaplan, A. (1982). A Generic Methodology for ThermographicDiagnosis of Building Enclosures. Public Works Canada. Report SeriesNo. 30.

Perseus. (2000). Survey 101—A complete guide to a successful survey(www.perseus_101b.htm).

Pike, P. (2001). Technology rant: I’m tired of feeling incompetent! PikeNetFebruary 11 (www.pikenet.com).

Schaefer, D., and Dillman, D. (1998). Development of a standard e-mailmethodology. Public Opinion Quarterly 62:378-397.

Schiller, G., et al. (1988). Thermal Environments and Comfort in OfficeBuildings. Berkeley: Center for Environmental Design Research,CEDR-02-89

Schiller, G., et al. (1989). Thermal comfort in office buildings. ASHRAEJournal October 26-32.

Sheehan, K., and McMillan, S. (1999). Response variation in e-mail sur-veys: An exploration. Journal of Advertising Research 39(4):45-54.

Swoboda, W., Muhlberger, N., Weitkunat, R., and Schneeweib, S. (1997).Internet surveys by direct mailing. Social Science Computer Review15(3):242-255.

Taylor, H. 2000. Does Internet research work? International Journal ofMarket Research 42(1):51-63.

USA Today (2001). Searching the Web in native language. February 27, p.7B.

Appendixes

63

Appendix A

Functionality and Serviceability Standards:Tools for Stating Functional Requirements and

for Evaluating FacilitiesFrançoise Szigeti and Gerald Davis, International Centre for Facilities

INTRODUCTION: THE FUNCTIONALITY ANDSERVICEABILITY TOOLS HAVE STRONGFOUNDATIONS1

The functionality and serviceability tools arefounded in part on “the performance concept in build-ing,” which has roots before World War II in Canada,the United States, and overseas. In the United States inthe 1950s and 1960s, the Public Buildings Service(PBS) of the General Services Administration (GSA)funded the National Institute of Standards and Tech-nology (NIST, then the National Bureau of Standards)to develop a performance approach for the procurementof government offices, resulting in the so-called PeachBook publication (NBS, 1971). Starting in the early1980s, the performance concept was applied to facili-ties for office work and other functions by the Ameri-can Society for Testing and Materials (ASTM) Sub-committee E06.25 on Whole Buildings and Facilities.Worldwide, in 1970, the International Council forBuilding Research Studies and Documentation (com-monly known as CIB) set up Working CommissionW060 on the Performance Concept in Building. In1982, the coordinator for that commission defined theconcept in those terms: “The performance approach is,first and foremost, the practice of thinking and work-ing in terms on ends rather than means. It is concernedwith what a building is required to do, and not with

prescribing how it is to be constructed” (Gibson, 1982).In 1998, the CIB launched a proactive program for theperiod 1998-2001 focused on two themes: theperformance-based building approach, and its impacton standards, codes and regulations, and sustainableconstruction and development.2

By 1985, the importance of distinguishing betweenperformance and serviceability had been recognized,and standard definitions for facility and facility service-ability were developed. Facility performance is definedby ASTM as the “behaviour in service of a facility fora specified use,” while facility serviceability is the“capability of a facility to perform the function(s) forwhich it is designed, used, or required to be used.” Bothdefinitions are from ASTM Standard E1480. Service-ability is more suited than performance to respondingto the stated requirements for a facility, because thefocus of performance is only on a single specified useor condition, at a given point in time, whereas service-ability deals with the capability of a facility to deliver arange of performance over time. In the InternationalOrganization for Standardization (ISO), related workhas been carried out within ISO/Technical Committee59/Sub-Committee 3 on Functional/User Requirementsand Performance in Building Construction.

The term programme, meaning a statement ofrequirements for what should be built, was in commonusage in the mid-nineteenth century by architecturalstudents at the Ecole des Beaux Arts in Paris and, there-after, in American universities as they adopted theFrench system. In North America, the architect’s basic

1For further information and details, see Szigeti and Davis,(1997) in Amiel, M. S., and Vischer, J. C., Space Design and Man-agement for Place Making Proceedings of the 28th Annual Confer-ence of the Environmental Design Resarch Association, The Envi-ronmental Design Research Association (EDRA), Edmond, Okla.,1997.

2CIB Pro-Active Program, see CIB Web site for further details at<www.cibworld.nl>.


services included architectural programming (i.e.,“confirming the requirements of the project to theowner”), but excluded setting functional requirements,which was the owner’s responsibility. In Britain andparts of Canada, the term “briefing” includes program-ming. By the mid-twentieth century, some clients forlarge or complex projects paid extra to have their archi-tects or management consultants prepare a functionalprogram for their projects.3 The functionality andserviceability tools were created to make it easier,faster, and cheaper to create such functional programsin a consistent and comprehensive manner, to linkrequirements to results, and to evaluate performanceagainst requirements.

There is now a worldwide trend toward the use of a“performance-based approach” to the procurement,delivery, and evaluation of facilities. This approach isuseful because it focuses on the results, rather than onthe specification of the means of production and deliv-ery. It reduces trade barriers and promotes innovationor at least removes many impediments to innovation.For such an approach to be successful however, thereis a need for more attention to be paid to the definitionand description of the purposes (demand-results), shortterm and long term, and for more robust ways of verify-ing that the results have indeed been obtained. This iswhy there is a mounting interest in building perfor-mance evaluations and other types of assessments. Cus-tomer satisfaction surveys, post-occupancy evaluations(POEs), lease audits, and building condition reports arebecoming more common.

This appendix has four sections: (1) context,(2) measuring the quality of performance of facilitiesusing the ASTM standards, (3) examples, and (4) finalcomments.

CONTEXT: PROGRAMMING AND EVALUATIONAS PART OF A CONTINUUM

Feed-Forward—The Programming-Evaluation Loop

Not only do most large organizations lack a compre-hensive facilities database, they also fail to develop an

institutional memory of lessons learned. They are toooften dependent on what best practices have been rec-ognized and remembered by individual real estate andfacility staff members and passed on informally to theirsubordinates and successors. Most often, such accu-mulated knowledge disappears with the individualsresponsible. Instead, as each facility project is acquired,whether it is new construction, remodel or refit, orleased or owned, both the facility and the processesinvolved should be evaluated. Each phase of eachproject should be considered a potential source oflessons, including planning, management, program-ming, design, construction, commissioning, occupancy,operation, and maintenance, even decommissioning.Figure A-1 shows such an ongoing cycle of feed-forward from project to project.

To be effective, such evaluations, or programs oflessons learned, need a way to organize the informa-tion and to relate and compare it to what the clientrequires now and in the future. Since 1965 whenTEAG—The Environmental Analysis Group/GEMH –Groupe Pour l’Etude du Milieu Humain—waslaunched, a programming assignment normally startswith an evaluation of the current facilities used by theclient or similar surrogate facilities if need be. Theseevaluations give invaluable information and serve as acontext for the programming process. The work thenproceeds with interviews of senior managers aboutcurrent problems and future expectations, and groupinterviews with occupants at several levels of the orga-nization. Questions are asked about what works, notjust what does not work. It is important to know whatshould be carried over from the current situation.

Over the years, interview guides and recording docu-ments have been developed for such evaluations. Thiswork and experience provided the foundation for thefunctionality and serviceability tools. Thus, evaluationsfeed into functional programs, which become the basisfor the next evaluations.

Defining Requirements

The functional program should focus on aspects ofthe project requirements that are important for theenterprise, in order to direct the best allocation ofresources within the given cost envelope. The objec-tive is to get best value for the users and owners. Aknowledgeable client will prepare, in-house or with thehelp of consultants, a statement of requirements (SOR),including indicators of capability of the solution that

3The first stand-alone, general practice in building programming,not part of an architectural or management consulting practice, wasTEAG—The Environmental Analysis Group, founded in 1965 byGerald Davis. The International Centre for Facilities was foundedin 1986 to focus on research and development activities related tofacilities and on standard development activities.

APPENDIX A 65

are easy to audit and are as unambiguous as practi-cable. This is an essential step of the planning phase fora project.

Portfolio management provides the link betweenbusiness demands and real estate strategy. At the port-folio level, requirements are usually rolled up and re-lated to the demands of the strategic real estate plan insupport of the business plan for the enterprise(Teicholz, 2001).

Requirements for facilities needed by an enterprisewill normally be included in a portfolio managementstrategy. An asset management plan for a facility wouldinclude the specific requirements for that facility. Astatement of requirements, in one form or another, moreor less adequate, is part of the contractual documenta-tion for each specific procurement.

Statements of requirements serve as the startingpoint for providers of material, products, facilities, ser-vices, and so forth. As experienced readers of this re-port likely know, if there is ever litigation or other li-ability issue, then the statement of requirements is thefirst document that the parties will turn to. It is the ref-erence point for any review process during tendering,design, production, and delivery, as well as for laterevaluation(s), no matter which methodology is used. It

is particularly necessary for performance-based anddesign-build procurement and for any project devel-oped using an integrated project team approach. In themore traditional approach, the contractual documentsnormally include very precise specifications (“specs”).In the experience of expert witnesses, the root of manycourt cases and misunderstandings can be traced backsquarely to badly worded, imprecise, incomplete state-ments of requirements that do not include any agreedmeans of verifying whether the product or service de-livered is in fact meeting the stated requirements.

In a performance-based approach and for design-build and similar procurements, the focus is on the ex-pected performance, or on a range of performances, ofthe end product. Therefore, the heart of these nontradi-tional approaches is defining those expected results andthe requirements of the customer or user in an objec-tive, comprehensive, consistent, and verifiable manner.In any dispute, it is necessary to be able to go back tothe contract and have a clear definition of what wasagreed between the parties. If the “legal” name of thegame is a “warranty of fitness for purpose,” then thepurpose has to be clearly spelled out, as well as theways to verify that “fitness.” This point is developedfurther later in this appendix.

FIGURE A-1 Feed-forward.

Projector Facility

A

Design

ConstructCommission

Evaluate

Projector Facility

B

Design

Evaluate

Portfolio and Asset ManagementProperty Operations and Maintenance

Best Practices Feed-Forward Best Practices Feed-Forward

User + ProjectRequirements

LessonsLearned

LessonsLearned

ConstructCommission

Based in part on a diagram by John Zeisel Diagram by Francoise Szigeti and Gerald Davis© 2000-2001 International Centre for Facilities, Inc.

Manage+ Use

Manage+ Use

RevitalizeEvaluate

RevitalizeEvaluate

ReviewReview

Nextiteration

User + ProjectRequirements


Life Cycle of Facilities, Shared Data, andRelationship to the Real Estate Processesof the Enterprise

For each facility, the information included in theasset management plan, plus the more detailed pro-gramming data and the financial data, are the founda-tion for the cumulative knowledge base of shared dataand support data about the facilities diagrammed at thecenter of Figure A-2. Throughout the life cycle of afacility, many people, such as portfolio and facilitymanagers, users, operations and maintenance staff, fi-nancial managers, and others, should be able to con-tribute to and access this pool of data, information, andknowledge.

Today, these kinds of data and information are stillmostly contained in “silos,” with many disconnectsbetween the different phases of the life cycle of a facil-ity. Too often the data are captured again and again,

stored in incompatible formats and difficult to corre-late and keep accurate. The use of computerized data-bases and the move to Web-based software applica-tions and projects are steps toward the creation of ashared information base for the management of realestate assets. Once such shared databases exist, thevalue of evaluations and benchmarking exercises willincrease because the information will be easier to re-trieve when needed. The shared data and knowledgebase will also make it easier to “ close the loop” andrelate the facilities delivered to the demands of the en-terprise.

In discussions at the Facilities Information Councilof the National Institute of Building Sciences, such re-creation of data over and over again has been identifiedas a major cause of wasted dollars and the source ofpotential savings. More important will be the reduc-tions in misunderstandings, the increased ability to pin-

OverallEnterprise

Based in part onDana K. Smith diagram for FIC of NIBS, 2000

Portfolio Management

Asset and Property Management

Project Delivery

Portfolio and ProjectRequirements

Decision: Keep/Dispose/Re-use

Evaluate

Operate andMaintain

Evaluate

Commission

Design

Program

Plan

Construct

Repair andAlterations

Renovate/Re-use

© 2000 International Centre for Facilities

Diagram byFrancoise Szigeti and Gerald Davis

Evaluate

Information Baseof Shared Data

and Support Data

Gap Analysisof PortfolioDemand

FIGURE A-2 Life cycle of facilities.

APPENDIX A 67

FIGURE A-3 Strategic to in-depth evaluations. Source: Francoise Szigeti and Gerald Davis, © 1999, 2000 InternationalCentre for Facilities.

Roll-up of focused in-depth data, such as

Building Condition Audit, LEED, etc.

In-D

epth

inve

stig

atio

ns

Ele

ctro

mag

netic

Ene

rgy

cons

erva

tion,

e.g:

BE

PA

C, B

RE

EA

M

Wat

er a

nd w

aste

man

agem

ent

PO

E -

Pos

t Occ

upan

cy

Fin

anci

al a

naly

sis

and

audi

t

Sec

urity

: T

hrea

t and

risk

anal

ysis

Hea

ting,

Ven

tilat

ing,

and

Air

Con

ditio

ning

Sys

tem

Ele

ctric

al a

nd e

lect

roni

csy

stem

s

Plu

mbi

ng a

nd p

ipin

gsy

stem

s

Roo

f sui

tabi

lity,

O&

M -

Ope

ratio

n an

dM

aint

enan

ce

An

d m

any

oth

erty

pes

of

in-d

epth

inve

stig

atio

ns

. . .

Strategic review of key indicators

Macro level scan, not in-depth investigation

drai

nage

, con

ditio

n

radi

atio

n

Indo

or A

ir Q

ualit

y

Life

cyc

le c

osts

and

valu

e en

gine

erin

tg

Life

cycl

e an

alys

is

Rea

l Est

ate

mar

ket

anal

ysis

Eva

luat

ion

point weak links in the information transfer chain, andthe improvements in the products and services becausethe lessons learned will not be lost.

For evaluations to yield their full potential as part ofthe life-cycle loop, the information that is fed forwardfrom such activities needs to be captured and presentedin comparable formats. Accepted terminology, stan-dard definitions, and normalized documentation willmake such comparisons much easier. The links be-tween evaluations and stated requirements should beexplicit and easy to trace.

Figure A-2 diagrams the life cycle of a facility, in-cluding the particular points in the cycle when mostevaluations occur. It shows in greater detail how one ofthe feed-forward loops unfolds. During project deliv-ery, of course, there are or should be evaluation loopsthat cannot be shown in the overall diagram.

Evaluations: From Strategic to In-Depth

Evaluations can happen at any time and can be trig-gered by many situations. They range in their approach

from the perceptual to the performance-based to thespecific and technical. At the moment, there is not yetany consensus as to what kinds of evaluations shouldbe done and when or how they should be done. Eventhe terminology is in great flux. Many terms are used,such as analysis, assessment, audit, evaluation, investi-gation, rating, review, scan, and so forth, usually to-gether with some qualifier, such as building conditionreport, building performance evaluation, facility as-sessment, post-construction evaluation, post-occu-pancy evaluation, serviceability rating, etc.

The range of tools, methods, and approaches toevaluations is quite wide as well as deep. In some situ-ations, it is appropriate to take a broad strategic viewand to use tools that can give answers quickly and withthe minimum of effort. At the other extreme, there aresituations that call for in-depth, specific, narrowly fo-cused, very technical engineering audits that can takeweeks and require sophisticated instrumentation. Fig-ure A-3 shows the relationship between these differentlevels of precision.


MEASURING AND MANAGING THE QUALITY OFPERFORMANCE OF FACILITIES USING THEASTM STANDARDS ON WHOLE BUILDINGFUNCTIONALITY AND SERVICEABILITY

Assessing Customer Perception andQuality of Performance

Assessing customer perception and satisfaction, orevaluating the quality of the performance delivered bya facility in support of customer requirements are twocomplementary, but not identical, types of assessments.In a recent issue of Consumer Reports, there is a seriesof items dealing with the ratings of health maintenanceorganizations. In one of the articles, the question of thequality of the ratings is posed and an important point ismade. “Satisfaction measures are important. But, don’tconfuse them with measures of medical quality…”(Consumer Reports, 2000). The key point is thatmeasuring customer satisfaction is important and nec-essary, but not sufficient.

There is a need for measuring the actual quality andperformance of the services and products delivered,whether it be medical care or the facilities and servicesthey provide in support of the occupants and theenterprise.

Defining Quality

Quality is described in ISO 9000 as the “totality offeatures and characteristics of a product or service thatbear on its ability to satisfy stated and implied needs”(ISO, 2000). Quality is also defined as “fitness for pur-pose at a given cost.” The difference between Tiffanyquality and Wal-Mart quality does not need to beexplained. Both provide quality and value for money.Both are appropriate, depending on what the customeris looking for, for what purpose, and at what price.

Quality therefore is not absolute. It is the mostappropriate result that can be obtained for the price oneis willing to pay. Again, in order to be able to evaluateand compare different results or offerings, and verifywhether the requirements have been satisfied, thesemust be stated as clearly as possible.

Measuring and Managing the Quality of Performance

Many enterprises, public and private, review theproject file during commissioning, or later, and notewhether the project was completed within budget and

on schedule. Some do assess how well each new orremodeled facility meets the need of the business userswho occupy it. Essential knowledge can be captured aspart of a formal institutional memory of what workswell, what works best, and what should not be repeated.There are an array of different methods and tools thatcan be used to capture this information. A number ofthose tools have been catalogued by a group of researchersand practitioners based at the University of Victoria atWellington, New Zealand (Baird et al., 1996).

A quality management (or assurance) program needsto measure and track performance against “statedrequirements.” Those who provide a product or service(e.g., a facility and its operations and management),should ascertain the explicit and implicit requirementsof the customers (occupants), decide to what level thoseneeds should be met, meet that level consistently, andbe able to show that they are in fact meeting thoserequirements within the cost envelope.

Such programs, therefore, need to start with anappropriate process for preparing statements of require-ments. These should include the ability to determineand assess features and characteristics of the product orservice considered; to relate them directly to customers’needs, expectations, and requirements; and to docu-ment it all in a systematic, comprehensive, and orderlymanner. Such documentation should include the meansto monitor compliance during all phases of the life cycleof the facility. When dealing with facilities, informa-tion should also be included about how the enterprise isorganized and its business strategy, and about expecta-tions related to quantity, constraints, environmental andother impacts, time, costs, and so forth. All theseelements have to be taken into consideration when con-ducting an overall evaluation, in particular at the timeof commissioning or shortly thereafter.

Using the ASTM Standards on Whole BuildingFunctionality and Serviceability to Measure Quality

The information provided by most POEs and by cus-tomer satisfaction surveys is primarily about occupantperception and satisfaction, which is often necessarybut rarely sufficient. It is seldom specific enough to beacted upon directly. Similarly, in-depth and specifictechnical evaluations usually do not address topicsdirectly related to the functional requirements of theusers or cannot be matched to those requirements.

Based on some 30 years of experience with both pro-gramming and evaluation, over the period 1987-1993,

APPENDIX A 69

the International Centre for Facilities (ICF) team cre-ated a set of scales that have now become the ASTMstandards on whole building functionality and service-ability, recognized as American National Standards(ANSI). More importantly, these standards are basedon a methodology for creating such scales that is cur-rently being balloted at the international level underthe authority of ISO TC 59/SC3 on Functional/UserRequirements and Performance in Building Construc-tion (ASTM, 2000).

These standards do provide information that can beacted upon. They measure the quality of services deliv-ered by the facility in support of the occupants as indi-viduals and as groups. The results from serviceabilityratings complement POEs and can be cross-referencedto customer satisfaction surveys (see examples in“Final Comments”). These standards currently provideexplicit, objective, consistent methods and tools andinclude the means to monitor and verify compliancewith respect to office facilities. The usefulness of suchstructured information goes beyond a single project. Itcan also be used for lessons-learned programs and forbenchmarking. The methodology could be appliedequally well to create a set of tools for measuring thequality of performance of any capital asset, includingall types of constructed assets, whether public infra-structure such as bridges, roads, and utilities, orbuildings.

This new generation of tools gives real estate pro-fessionals the means to evaluate the “fit” betweenfacilities and the users they serve. These tools use indi-cators of capability to assess how well a proposeddesign, or an occupied facility, meets the functionalrequirements specified by the business units andfacility occupants. Even a small business, with only afew dozen staff, needs to capture and convenientlyaccess the key facts about its workplaces, how they areused, and lessons to apply “next time.”

Functionality and Serviceability:Matching User Requirements (Demand)and Their Facilities (Supply)

Evaluations are most useful when they provide themeans to compare results to expectations. Figure A-4shows the use of bar-chart profiles to match functionalrequirements and serviceability ratings using theASTM standard scales.

ASTM Standard Scales

The ASTM standard scales provide a broad-brush,macro level method, appropriate for strategic, overalldecision-making. The scales deal with both demand(occupant requirements) and supply (serviceability ofbuildings) (MacGregor and Then, 1999). They can beused at any time, not just at the start of a project. Inparticular, they can be used as part of portfolio man-agement to provide a unit of information for the assetmanagement plan, on the one hand, and for the roll-upof requirements of the business unit, on the other.

The ASTM standard scales include two matched,multiple-choice questionnaires and levels. One ques-tionnaire is used for setting workplace requirements forfunctionality and quality. It describes customerneeds—demand—in everyday language, as the core offront-end planning. The other, matching questionnaireis used for assessing the capability of a building to meetthose levels of need, which is its serviceability. It ratesfacilities—supply—in performance language as a firststep toward an outline performance specification.

Both cover more than 100 topics and 340 buildingfeatures, each with levels of service calibrated from 0to 9 (less to more). These standard scales are particu-larly suitable as part of the front end for a design-buildproject, to compare several facilities on offer to buy orlease. The scales can also be used to compare the rela-tive requirements of different groups.

This set of tools was designed to bridge between“functional programs” written in user language on theone side and “outline specifications and evaluations”written in technical performance language on the other.Although it is a standardized approach, it can easily beadapted and tailored to reflect the particular needs of aspecific organization.

For organizations with many facilities that housesimilar types of functions, the functionality and service-ability scales capture a systematic and consistent recordof the institutional memory of the organization. Theiruse speeds up the functional programming process andprovides comprehensive, systematic, objective ratingsin a short time.

The Serviceability Tools and Methods(ST&M) Approach

The ST&M approach (Davis et al., 1993) includesthe use of the ASTM standards, and its results, but also


FIGURE A-4 Matching demand and supply—gap analysis. Source: Francoise Szigeti and Gerald Davis, © 1999, 2000International Centre for Facilities.

FunctionalityRequired

Serviceabilityof Facility

Synthesize userrequirements

Showrequiredlevels as abarchart

9

7

5

3

1

• Set required levelsof serviceability, byusing the standardmultiple-choicequestionnaires.• Set relativeimportance and setminimum threshold, ifany.

Rate capabilities offacility to support

users

9

7

5

3

1

Set rating levels ofserviceability by

using the standardmultiple-choicequestionnaires

Show ratinglevels asbarcharts

Compare strengths and shortfalls.Consider relative importances.

Consider tradeoffs, then decide.

provides formats for describing the organization, func-tion-based tools for estimating how much floor area anorganization needs, and other tools necessary to pro-vide needed information for the statement of require-ments (SOR).

At the heart of this approach is the process of work-ing with the occupant groups during the programmingphase of the project cycle, as well as during any evalu-ation phase. This process of communication betweenthe providers of services and products (in-house andexternal) and the other stakeholders (in particular theoccupants), of valuing their input, and of being seen tobe responsive can be as important as the outcome itselfand will often determine the acceptability of the results.

This is where satisfaction and quality overlap.ST&M includes several kinds of methods and tools,along with documents and computer templates forusing them:

1. Functional requirement bar-chart profile andfunctional elements

2. Facility serviceability bar-chart profile and indi-cators of capability

3. A match between two profiles and comparisonswith up to three profiles

4. A gap analysis and selection of “strength and con-cerns” for presentation to senior management

5. Text profiles for use in a statement of require-ments and equivalent indicators of capability

6. Descriptive text about the organization, its mis-sion, relevant strategic information, and otherinformation about the project in a standard format

7. Quantity spreadsheet profiles8. Building loss features (BLF) rating table9. Footprint and layout guide

The ASTM standards and the ST&M approach areproject independent. Requirements profiles can be pre-pared at any time, and serviceability ratings can be doneand updated at a number of points over the life cycle ofa facility.

APPENDIX A 71

How Do These Tools Fit in the Overall CorporateReal Estate Framework?

Setting requirements and evaluating results are twoparts of what should be an ongoing dialogue betweenusers and providers. Evaluations are becoming an in-dispensable tool for decision-making by senior man-agement and for appropriate responsiveness by allthose involved, whether they are working in-house orare external providers.

Facilities are an important resource of the enterprise.There are three main processes to take into account:(1) demand; (2) management, planning, procurement,production, and delivery; and (3) operations, mainte-nance, and use.

Figure A-5 shows how the different processes in-volved relate to each other and to the enterprise.

In Figure A-6, the functionality and serviceabilityscales are shown as an overlay on this framework. Inthis manner, it is possible to see how they relate to theunderlying corporate real estate processes. The pro-cesses diagrammed here are each complex sets ofactivities and secondary processes. A detailed map ofsuch activities is included in the next volume of scalesto be published by the ICF (Davis et al., in press).

Figure A-6 also shows the relationship to a new setof scales prepared to rate the condition and estimatedresidual service life of a facility, to compare them tothe needs of the enterprise. These scales are also used

FIGURE A-5 Corporate real estate processes—linking to the enterprise. Source: Francoise Szigeti and Gerald Davis, © 1999,2000 International Centre for Facilities.

External Culture and Strategic Environment

Other Stakeholdersand Investors

CRE Knowledge base

AssetAcquisition -

Project Delivery

CRE&FMInformation Base

Diagram byFrancoise Szigeti and Gerald Davis© 1999, 2001 International Centre for Facilities

EnterpriseInformation

Infrastructure

Issues of sustainability, health,productivity improvements and otherbenefits of real estate for occupants,

customers and others

USEby Occupants

and Others

Work-place

Demand

EnterpriseFinancialStrategy

EnterpriseOperations

OverallEnterpriseStrategy

OverallEnterprise

Information flows

Legend:

Primary processesfor real estate

Provide facilities

Decision flows:CRE&FMEnterprise

Portfolio / StrategicAsset Management

ProjectRequire-ments

OverallPortfolio

ManagementPlan

AssetManagement

Plans

Budgetcycle

process

RealEstate

Strategy

Focus on managementof overall portfolio

Focus on managementof individual assets

OperationalAsset

Management

FacilityOperations &Maintenance


FIGURE A-6 Corporate real estate processes and use of the serviceability tools. Source: Francoise Szigeti and Gerald Davis,© 1999, 2000 International Centre for Facilities.

OverallEnterprise

Estimates ofWork-place

Demand

Useof facilities

for EnterpriseOperations

Time

WorkplaceSuitability

Condition& Service

LifeSuitability

Details ofCondition

Condition &Service Life

Rating

Condition &Service LifeRequirement

FunctionalityRequirement

ServiceabilityRating

Diagram by Francoise Szigeti and Gerald Davis© 1999, 2001 International Centre for Facilities

Asset AcquisitionProject Delivery

Requirements

OperationalAsset

Management

FacilityOperations &Maintenance

AssetSuitability

Portfolio / Strategic Asset Management

OverallPortfolio

ManagementPlan

AssetManagement

Plans

Budget cycleprocess

Priorities amongrequirements

RealEstate

Strategy

Focus on managementof overall portfolio

Focus on managementof individual assets

for setting budget priorities for repair and alterationprojects.

EXAMPLES: USES OF THE ASTM STANDARDSAND LINKS TO OTHER TOOLS

Link to GSA’s Customer Satisfaction Survey:GSA, John C. Kluczynski Building, Chicago

GSA regularly assesses the satisfaction of occupantsof its major buildings, using a version of the customer

satisfaction survey developed by the InternationalFacility Management Association. The satisfactionlevels of occupants of its landmark John C. KluczynskiBuilding in downtown Chicago were compared tolevels in a serviceability rating of the building and tofunctionality requirement profiles for the main catego-ries of occupant groups.

The results correlated closely. The serviceabilitylevels both predicted and explained the satisfactionlevels. However, the customer satisfaction survey hadmore detail about how occupants felt about the speed

APPENDIX A 73

and thoroughness with which building operations staffresponded to problems and complaints. The service-ability scales gave more information about actualstrengths and concerns of the facility to meet occupantfunctional needs.

Together, these two complementary studies pro-vided needed supporting information to submissionsfor the funding of several renovation projects. Anexample of a bonus from the functionality and service-ability project was the identification of ongoingsecurity concerns of the staff. Occupants did not real-ize that the situation had been remedied and that theftshad been reduced by three-quarters since the assign-ment of a policeman in uniform on-site, patrolling thecorridors. On the basis of this particular finding, a com-munications and public relations effort was launchedto inform the staff of the beneficial impact of the pres-ence of the uniformed policeman.

Link to Prior POEs: U.S. State Department

During the 1990s, the U.S. State Department con-ducted POEs after many major projects. The findingsfrom these projects were analyzed by in-house staff andothers of the Office of Foreign Buildings Operations(now Office of Overseas Buildings Operations). Then,in the late 1990s, a functionality requirement profilewas developed for chanceries, using the ASTM stan-dard scales. Data, as well as insights, from the POEswere taken into account in setting requirement levelsand in preparing the main requirement profile for a basebuilding as well as for the variant profiles for the dif-ferent zones in a chancery. The ASTM standard scalesprovided a structure for applying the information fromthe POEs, which could then be directly related toequivalent levels of serviceability.

Part of Asset Management Plans: Public Works andGovernment Services Canada

When the serviceability tools and methods were firstdeveloped, the government of Canada rated the service-ability of all its major office buildings across thecountry. Recently, it has issued contracts to update theasset management plans of all its major office build-ings. Each plan is required to contain a serviceabilityrating using the standard serviceability scales.

Part of Tool Kit for Portfolio Management andSetting Budget Priorities for Repair and AlterationProjects: PG&E

Pacific Gas & Electric Corporation (PG&E) hasrealigned the way it plans and manages its portfolio ofreal estate and sets priorities in its annual budget forrepairs and alterations. Integral to its new manage-ment approach are the ASTM standard scales, theserviceability tools and methods including the newscales for building condition, and estimated remainingservice life. These ratings are compared to the requiredlevel set for that facility.

New Design: National Oceanic and AtmosphericAdministration (NOAA)

The groups that operate the weather satellites ofNOAA needed a new headquarters building. Theirfunctional requirements were specified using theASTM standard scales. Their requirement profile wasthen compared to that of private sector organizationsdoing similar work, such as the headquarters of a gaspipeline company or the headquarters of a mobilephone company. NOAA’s requirement profile was verysimilar to what was needed by other organizationsdoing similar kinds of work; there were very few dif-ferences. This provided a kind of benchmarking forNOAA’s senior management and showed that NOAA’srequirements were appropriate and consistent withprivate sector practice, even though they were muchmore demanding than would typically be provided fora general administrative office in government.

Choosing a Lease Property: U.S. State Department,Passport Office

The U.S. State Department and GSA reached agree-ment on the functionality requirement profile for itspassport offices where citizens can come to have theirapplications adjudicated and a passport issued quickly.When new leased office space was needed for its pass-port office in New Orleans, the requirement profile wasverified for its applicability to this particular office.Then about a quarter of the requirement scales wereused to scan the properties on offer that GSA had iden-tified as relevant. In two days, six properties werescanned for serviceability levels. Only one out of six


was found to meet the essential functionality require-ments. However, once the real estate manager in GSAunderstood the requirement profile of the occupantgroup, she was quickly able to identify two other validoptions. Within weeks, the “best fit” was identified,and a lease negotiated.

Requests for Proposals (RFPs) and Design Reviews:State Department Chanceries andHigh-Tech Organizations

During 1999 and 2000, the U.S. State Departmentused its functionality requirement profile and service-ability indicators to assess the functionality of designproposals for new embassies and consulates to bedeveloped in the sequential project developmentprocess, design-bid-build. The department’s require-ment profile was also used to assess and compare thefunctionality of proposals using the integrated, design-build procurement process. In some cases, proposalshad very similar levels of functionality, while someother proposals showed significant differences betweenthe functionality of the proposals and what had beenrequired.

Thereafter, the same requirement profile was usedduring design reviews as a benchmark to ensure thatthe designs were continuing to respond to the require-ments stated in the RFP. Contractors were trained to dothese assessments, to create comparisons bar-charts,and to analyze the gaps between the designs and therequirement profile.

When a slow-growth, high-tech corporation neededa new corporate headquarters, it developed a functionalrequirement profile in the language and format of theASTM standards and included it, verbatim, in its RFP.Responses to the RFP were rated, using the service-ability scales. Although a number of the proposals werefairly tightly clustered on price, there was a significantdifference in functionality among the proposals.

Levels of Service for Outsourcing:A Major Oil Company

When a major oil company was considering out-sourcing its facility management operations, it askedall companies who proposed to base their cost proposalson using the same levels of serviceability, as specifiedusing the ASTM standard serviceability scales. Seniormanagement also asked the “in-house provider” to self-rate using the same standards. Both the in-house facili-

ties group and the bidders were able to do this on shortnotice, without having received any special training orguidance in using the standards. Comparable proposals,based on these consistent requirements, were receivedon schedule without difficulty. This company alsoasked TEAG to rate its main campus and to prepare aprofile of requirement for the largest occupant grouphoused at that campus. This third-party assessmentserved as a benchmark to compare the results from theother assessments and proposals.

FINAL COMMENTS

Some Relevant Anecdotes

The value of a property and its long-term benefitsare not just a matter of real estate dollars and cents andtechnical performance; corporations also look at theeffectiveness of the workplace for core business opera-tions and at the strategic advantage that facilities canprovide. Successful facilities and real estate groupsunderstand this (FFC, 1998). It can cost or earn thecompany far more than the rise or fall of propertyprices. For example, one vice president for facilities atan aircraft company explained to us that facilities costsrepresent about 5 percent of the total cost of each air-plane sold, but that 5 percent is critical to the ability ofhis company to deliver new planes on time and onbudget. If a new hangar is not ready on time or thefacilities get in the way, the whole production line canbe delayed or grind to a halt. The same holds true forsmaller companies who rent office space. For them thecost of rent, utilities, and other charges runs at aboutthe same percentage.

For organizations, big or small, a 1 percent increase(or decrease) in the productivity of core business opera-tions, brought about by an inadequate workplace, isprobably at least 10 times greater than a 1 percentincrease (or decrease) in the value of the real propertyconsidered as a real estate asset. Put another way, hereis the example of a laboratory where facilities wereunderused and inefficient: At zero facilities cost andwith minimum rearrangements, an extra 15 scientistscould be added. This would give the lab extra grossrevenue while lowering the cost of square feet peremployee. On the other hand, more substantial changesin the facility layout would allow the lab to nearlydouble its population. This retrofit would cost far lessthan the cost of a new facility. Other functionalimprovements would increase the effectiveness of the

APPENDIX A 75

staff by about 15 percent and speed time to market byabout six months. All of these proposed changes werebased on an assessment of functional capability. Over-all, the asset value in use was expected to increase bymore than $3,000,000, after taking into account retrofitcosts. The earning power of the lab would be multi-plied by almost 2.5. These numbers were calculated byone of the major accounting firms. Thus, typically, thegreatest leverage for a facility comes from enhancingthe performance of the core business. A factor of 10 ormore is not unusual.

The effect of a facility on the health of its occupantscan have a severe impact on productivity. Medicalrecords are seldom used to prove this point but prob-ably could be used more often. On one occasion, ICFwas allowed to use records of sick days as part of acomprehensive facility evaluation after a major con-solidation of staff into a single facility. After plottingthe sick leave information for each of 18 groups for theyear prior to the move, the year of the move, and thefirst year after the move, it became apparent that for allbut two groups, the curve shifted up. For two groups,the curve shifted down. For those two groups, the build-ing they came from was worse than the new facility. Itwas estimated that the number of days lost to theincreased sick leave and a few other facility-relatedfactors amounted to more than the annualized first costof the building.

Sometimes, the effect of the facility can be drasticand immediate. In one case, due to some work beingdone in one part of the facility, traffic was redirectedalong an internal corridor cutting through the “terri-tory” of a work group. What had been a “private path”was transformed overnight into a “major highway”(Davis and Altman, 1976). The partitions around thatgroup were glass above 1 meter, which allowedpassers-by to see into the space. ICF had warned thatsuch a situation should not be allowed to happenbecause of the “fish-bowl” effect. The group in ques-tion was working on a critical path product that was atthe heart of the future of the company and still highlysecret. The group simply stopped work and did not putwork on their desks. When the ICF team arrived on-site that day, it was asked to come directly to the officeof the senior manager responsible for facilities. A workcrew was commandeered to work overnight. Butcherpaper was pasted over the glass to create visual pri-vacy. By the next morning the problem had beensolved, and work resumed.

In another case, staff retention was the victim. Amajor industrial corporation was recruiting youngengineers to replenish its aging population, but thesewere leaving the company in record numbers after threemonths or less on the job. The human resources depart-ment conducted exit interviews to find out what theproblem was. The young recruits reported that thespace they were asked to work in was so unpleasantand antiquated that they felt the company had no regardfor them. The job market at that time was good, andthey could get jobs at other companies offering compa-rable salaries and much more attractive and modernfacilities. This caused the company to start a $300 mil-lion rehabilitation program of its offices and to gener-ally pay more attention to the physical setting of work.

Facilities have an impact when attracting staff,which can be the reverse of the last anecdote. Anothermajor industrial company reported that it was locatedin an industrial area with other competitors. Its policywas to make its grounds attractively landscaped and toprovide each of its software engineers with a private,well-furnished office with a window overlooking treesand flowers. The human resources department at thatcompany could confirm that this “perk” was worthabout 10 percent of payroll, that is, more than the annu-alized cost of the buildings and grounds.

Current Developments and Trends

Scales for Rating Condition and Estimated RemainingService Life

As stated earlier, new scales have been developedby the ICF to enable a manager to set priorities forrepair and alteration projects in the annual budget cycle.These scales are used for building condition, estimatedservice life, and asset management. They have beendesigned to assist managers to take into account theactual and required physical condition and estimatedremaining service life of a facility or of its main sys-tems and components. Typically, building conditionreports give cost estimates to return a building to itsoriginal design but do not link directly to the level offunctional support now required for occupant opera-tions. These new scales can be matched directly, forgap analysis against the condition requirement profilein the asset management plan and overall portfoliomanagement plan. They complement the informationabout the functional suitability of the facility to supportthe mission of the occupant group.


Scales for Other Building Types

A set of scales, similar to the ASTM standards, hasbeen drafted for low-income housing and is beingtested in New Zealand (Gray, 2001).

ICF has just completed additional scales for serviceyards and maintenance shops. Scales are also beingdeveloped to better cover sustainable building, manu-facturing, retail, laboratories, education, health care,courts, and so forth.

Integrated Tools for Performance-Based Procurement

The Dutch government has mandated that all publicprocurement be performance based. With the ASTMstandards as a starting point, the Dutch BuildingAgency has developed a systematic approach to defineclient expectations for total building performance (Anget al., 2001). This approach also relates the translationof “inputs” and “outputs” at different phase of theproject delivery process.

Strategic Asset Management

In some countries, portfolio management and strate-gic planning come under the term “strategic asset man-agement” (SAM). In these places, asset is the preferredterm, rather than facilities or buildings, of public sectormanagers who deal with all kinds of constructed capitalassets, many of which may not be buildings. One of themost pertinent publications on the subject is a news-letter dedicated to the international review of all areasof performance and strategic management of assets,including economic considerations. Linking perfor-mance evaluations and costs is still a tricky business.Some of the concepts, such as profiles, levels of ser-vice, benchmarking targets, etc., which are becomingpart of the performance evaluation, are explained in thenewsletter in a practical and approachable way, with anemphasis on sharing of experience (Burns).

Levels of Service, Performance Profiles,Performance Benchmarks

The use of levels, or targets, is becoming prevalentfor outsourced contracts, for performance-based pro-curements, and for strategic planning. The Departmentfor Administrative and Information Services of thestate of South Australia, is currently developing a build-ing performance assessment and asset development

strategy, which adapts some of the ASTM standardscales and methodology to its own circumstances.

Warranty of Fitness for Purpose—Duty of Care VersusDuty of Results

In the wake of ISO 9000, with the advent of newintegrated team approaches to projects, and performance-based procurement approaches, such as design-build,and construction manager at risk, there is an increasedawareness of the need to state requirements more pre-cisely and comprehensively and to be able to confirmthat the resultant asset meets those requirements. Fur-ther, when delivering a full package, the contractor hasthe legal responsibility to deliver a product that fits theintended purpose. This is a major change for the tradi-tional legal concepts of professional liability and dutyof care, based on professional competence and acceptedpractice. This legal territory is being explored bygroups such as the Design-Build Institute of Americaand the CIB (CIB, 1996).

In Conclusion

Evaluations are here to stay and will likely be takenfor granted in the not too distant future. At a prior Fed-eral Facilities Council (FFC) Forum, the presentationof the Amoco common process, developed by itsWorldwide Engineering and Construction Division,included the following in Figure 2: “Operate—Evalu-ate asset to ensure performance . . . ” (FFC, 1998).

To be more effective and useful, evaluations willhave to be better coordinated with the information con-tained in statements of requirements. At the same FFCforum, several presenters included a project system intheir presentation. The presentation by the director-construction of The Business Roundtable included adescription of “Effective Project Systems” developedby the Independent Project Analysis Corporation ofReston, Virginia. He made the point that “the supplychain begins when the customer need is identified andtranslated into a business opportunity” (FFC, 1996). Insuch project systems, which usually are conducted byintegrated project teams, the evaluation of alternativesolutions is taken for granted. Thus, evaluations are partof the planning of the projects, not an afterthought.

The worldwide trend to deal with performance defi-nitions rather than prescriptive or deemed-to-satisfyspecifications will likely continue to spread. In the nextfew years, a further increase in the use of evaluations

APPENDIX A 77

will be driven by the acceptance of performance basedprocurement by the World Trade Organization (WTO),the European Union, and the European member coun-tries. Performance-based codes are also being adoptedin a number of countries, including the United States.

The European Union (EU) is following suit. Sincedealing with results rather than specifying solutionsmeans that these results need to be shown to be per-forming as required, there is an assured future forevaluations and for evaluations as part of the process atmany stages. Such developments, as they affect thebuilding industry, will be the focus of a major thematicnetwork being launched by the CIB, with EU funding(Bakens, 2001). This network will also include partici-pants from the United States and other countries out-side the EU. A key task will be how to prepare state-ments of requirements and their verification.

In the United States, a performance-based code hasbeen adopted as a component of the new Unified Inter-national Building Code that has brought the three majorcodes together. Work is continuing at ASTM, theAmerican Institute of Architects (AIA), NIST, and theGSA, to cite only a few of the key leaders. 4

Benchmarking, lessons-learned programs, con-tinuous improvements, and performance metrics arebecoming part of business as usual. Indeed, to quoteagain from the 1998 FFC report: “What are the charac-teristics of the best capital project systems? In additionto using fully integrated cross-functional teams, theyactively foster a business understanding of the capitalproject process. . . . The engineering and project man-agers are accountable to the business, not the plantmanagement. There are continuous improvementefforts that are subject to real and effective metrics”(FFC, 1998). The evolution of POEs into building per-formance evaluations, and now into an ongoing evalu-ative stance, is likely to become the accepted normbecause it is part of the best practices of companies thathave succeeded in using capital projects in support oftheir primary business.

ABOUT THE AUTHORS

Gerald Davis helps decision makers and facility man-agers implement solutions that enhance worker effec-tiveness, improve the management of portfolios ofcorporate real estate, solve facility-related problems,and ensure the optimum use of buildings and equip-ment. Since 1965, Mr. Davis has been considered apioneer and internationally recognized expert in strate-gic facility planning, facility pre-design, programming,performance-based evaluations, and ergonomics. Assenior author, he led the team that developed theServiceability Tools & Methods® used to define work-place and facility requirements and to rate existing andproposed facilities. Previously, he led the “ORBIT-2”project, a major North American multisponsor studyabout offices, information technology and user organi-zations and about the impact of each on the other. Hecoauthored the “1987 IFMA Benchmark Report”which was the first of its kind. His work has been pub-lished in numerous trade and professional journals andbooks. He is the recipient of the Environmental DesignResearch Association Lifetime Career Award (1996),the IFMA Chairman’s Citation, (1998), was named anIFMA fellow in 1999, and one of 50 most influentialpeople in the construction industry by the Ottawa Busi-ness Journal. He is an ASTM fellow, and CertifiedFacility Manager, president and chief executive officer,International Centre for Facilities (ICF), president,TEAG (The Environmental Analysis Group), chair,ASTM Subcommittee E06.25 on Whole Buildings andFacilities; past chair, ASTM Committee E06 on Per-formance of Buildings; and past chair, IFMA StandardsCommittee (1993-99). He is also the U.S. (ANSI) votingdelegate to the ISO Technical Committee 59 on Build-ing Construction, to its Subcommittee 3 on Functional/User Requirements and Performance in Building Con-struction, and the former delegate to its Subcommittee2 on Terminology and Harmonization of Language.Mr. Davis was recently appointed the Convenor ofWork Group 14 on Functional Requirements/Service-ability.

Francoise Szigeti is the vice-president of the Inter-national Centre for Facilities, Inc., a scientific and edu-cational public-service organization established toinform and help individuals and organizations improvethe functionality, performance, and serviceability offacilities. She is also vice president and secretary-treasurer of The Environmental Analysis Group

4ASTM Committee E06 on Performance of Buildings has over-sight over this whole subject matter. The AIA has a Center forBuilding Performance. NIST continues to provide leadership withrespect to housing and other more technical applications of the con-cept. The GSA, under the leadership of its Office of Government-wide Policy, Real Estate, has embarked on a major program ofresearch and publications on the subject.


(TEAG) - Groupe pour l’Etude du Milieu Humain(GEMH). She is president of Serviceability Tools &Methods, Inc. Ms. Szigeti is one of the vice-chairs ofASTM Subcommittee E06.25. She has served on theboard of the Community Planning Association ofCanada, the Environmental Design Research Associa-tion and the International Association for the Study ofPeople and Their Physical Surroundings. She is a co-author and member of the team that developed theServiceability Tools & Methods® used for definingworkplace and facility requirements and for ratingexisting and proposed facilities. Previously, shelaunched and participated in the “ORBIT-2” project, amajor North American multisponsor study aboutoffices, information technology, and the user organiza-tions, and about the impact of each on the other. She isa coauthor of the “1987 IFMA Benchmark Report”,which was the first of its kind. Ms. Szigeti attended theEcole Superieure d’Interpretes et de Traducteurs,Universite de Paris. She is the recipient of the EDRALifetime Career Award (1996).

REFERENCESAng, G., et al. (2001). A Systematic Approach to Define Client Expectation

to Total Building Performance During the Pre-Design Stage. Proceed-ings of the CIB 2001 Triennial Congress.

ASTM (American Society for Testing and Materials). (2000). ASTMStandards on Whole Building Functionality and Serviceability, ASTM,West Conshohocken, Pa.

Baird, G., Gray, J., Isaacs, N., Kernohan, D., and McIndoe, G. (1996). Build-ing Evaluation Technique. Wellington, New Zealand: McGraw Hill.

Bakens, W. (2001). Thematic Network PeBBu—Performance Based Build-ing—Revised Workplan. Rotterdam:CIB.

Burns, P. (Ed.) SAM—Strategic Asset Management Newsletter AMQ Inter-national, Salisbury, South Australia.

CIB Publication 192. (1996). A Model Post-Construction Liability and In-surance System prepared under the supervision of CIB W087.Rotterdam, Holland.

Consumer Reports (2000). Rating the Raters, August 31.Davis, G., and Altman, I. (1976). Territories at the work-place: Theory into

design guidelines. In: Man-Environment Systems, Vol. 6-1, pp. 46-53.Also published, with minor changes, in: Korosec-Serfati, P. (Ed.)(1977). Appropriation of Space, Proceedings of the Third InternationalArchitectural Psychology Conference Strasbourg, France: Louis PasteurUniversity.

Davis, G., et al. (1993). Serviceability Tools Manuals, Volume 1 & 2 Inter-national Centre for Facilities: Ottawa, Canada.

Davis, G. et al. (2001). Serviceability Tools, Volume 3—Portfolio and AssetManagement: Scales for Setting Requirements and for Rating the Con-dition and Forecast of Service Life of a Facility—Repair and Alteration(R&A) Projects. International Centre for Facilities: Ottawa, Canada.

Federal Facilities Council. (1998). Government/Industry Forum on CapitalFacilities and Core Competencies. Washington, D.C.: National Acad-emy Press, p. 19.

Gibson, E.J. (1982). Working with the Performance Approach in Building.CIB Report, Publication 64. Rotterdam, Holland.

Gray, J. (in press). Innovative, Affordable, and Sustainable Housing.Proceedings of the CIB 2001 Triennial Congress. Rotterdam, Holland.

ISO 9000, Guidelines 9001 and 9004. (in process of reedition).McGregor, W., and Then, D.S. (1999). Facilities Management and the Busi-

ness of Space. Arnold, a member of the Hodder Headline Group.National Bureau of Standards. (1971). The PBS Performance Specification

for Office Buildings, prepared for the Office of Construction Manage-ment, Public Buildings Service, General Services Administration, byDavid B. Hattis and Thomas E. Ware of the Building Research Divi-sion, Institute for Applied Technology, National Bureau of Standards.Washington, D.C.: U.S. Department of Commerce NBS Report 10 527.

Szigeti, F., and Davis, G. (1997). Invited paper. In: Amiel, M.S., andVischer, J.C., Space Design and Management for Place Making. Pro-ceedings of the 28th Annual Conference of the Environmental DesignResearch Association (EDRA). Edmond, Okla.: EDRA.

Teicholz, E. (Ed.) (2001). Facilities Management Handbook. MacGraw-Hill.

79

Appendix B

A Balanced Scorecard Approach toPost-Occupancy Evaluation:

Using the Tools of Business to Evaluate FacilitiesJudith H. Heerwagen, Ph.D., J.H. Heerwagen and Associates

In the past decade, organizations have begun to lookat their buildings not just as a way to house people butalso as a way to fulfill strategic objectives (Becker andSteele, 1995; Horgen et al, 1999; Ouye and Bellas,1999; Grantham, 2000). In part this is due to re-engineering and downsizing of the past two decades.Also, however, chief executive officers (CEOs) arebeginning to think of their buildings as ways to achievestrategic goals such as customer integration, decreasedtime to market, increased innovation, attraction andretention of high-quality workers, and enhanced pro-ductivity of work groups.

Traditional post-occupancy evaluation (POE) methodsdo not provide the type of feedback needed to assessthese organizational outcomes. POEs focus on individual-level assessment, most typically on satisfaction, usepatterns, and comfort, rather than on organizational- orgroup-level outcomes associated with core businessgoals and objectives. Because organizations areincreasingly asked to justify all of their major expenses,including facilities, evaluation methods that begin toaddress these higher-level issues would be of greatvalue. At the present time, there are very few data toshow linkages between facility design and businessgoals.

As a result, decisions are often made on the basis ofreducing costs. Current cost-focused strategies includereducing the size of work stations, moving to a universalplan with only a few work station options, eliminatingprivate offices or personally assigned spaces, andtelecommuting. Evaluation methods that identify andmeasure the business value of facilities would be ahighly valuable way to expand the current knowledge

base and to provide a wider array of outcomes againstwhich to measure facility effectiveness.

Ouye (1998; Ouye and Bellas, 1999) suggests thatworkspace design and evaluation can become morestrategic by adopting the tools of business, specificallythe Balanced Scorecard (BSC) approach proposed byKaplan and Norton (1996). As applied to facilities, theBSC approach pioneered by Ouye and the WorkplaceProductivity Consortium is valuable not only for evalu-ation purposes, but also for design because it forcesdesigners to think systematically about the relationshipbetween the workplace and organizational effective-ness. Although the Balanced Scorecard was developedprimarily with the private sector in mind, the approachis also applicable to the evaluation of governmentfacilities. A core theme for both the private sector andthe government is to provide facilities that are bothefficient and effective. As noted in the General ServicesAdministration’s (GSA’s) The Integrated Workplace(GSA, 1999, p.5):

By using the Integrated Workplace as part of your strategic develop-ment plan, matching business goals to workplace designs, you canconsolidate and reconfigure the spaces where you work while pro-viding people with the tools they need to support the organization’smission.

Even more important for federal facilities is thestrong link between the BSC approach and the require-ments for strategic planning and performance assess-ment initiated by the Government Performance andResults Act of 1993 (GPRA). GPRA was enacted aspart of the Clinton administration’s “ReinventingGovernment” initiative to increase the efficiency offederal agencies and to make them more accountablefor achieving program results. GPRA requires federal


departments and agencies to develop methods for mea-suring their performance against strategic goals andprogram objectives, an approach that is very consistentwith the Balanced Scorecard. Because the BSC focuseson evaluation as a means to enhance overall strategicperformance, the results from the BSC approach wouldprovide valuable input to the GPRA performancereview for federal facilities. At the present time, mea-sures of facility “success” include costs per square footof space and square foot per occupant. Such measuresdo not address the strategic issues of concern to GPRAand the Balanced Scorecard.

The Balanced Scorecard assesses four categories ofperformance: financial, business process, customerrelations, and human resource development (Kaplanand Norton call this dimension “learning and growth”).

The term “balanced” refers to several factors. First,there is a balance across the four categories to avoidoveremphasis on financial outcomes. Second, theevaluation includes both quantitative and qualitativemeasures to capture the full value of the design project.And third, there is a balance in the levels of analysis–from individual and group outcomes to higher-levelorganizational outcomes. Figure B-1 shows some pos-sible measures to use in each category. All of thesemeasures have logical links to the workplace environ-ment, and in some cases there is empirical support also.

This appendix draws on the framework developedby Ouye (1998; Ouye and Bellas, 1999) but expands itto include greater discussion of the links betweenphysical space and business outcomes. The approachand process described in this chapter also focus on dif-

FINANCIAL OUTCOMES

• Operating/maintenance costs

• Costs of building related litigation

• Resale value of property

• Rentability of space

BUSINESS PROCESS OUTCOMES

• Process innovation

• Work process efficiency

• Product quality

• Time to market

STAKEHOLDER RELATIONS

• Public image and reputation

•

Customer satisfaction

•

Community relationships

HUMAN RESOURCE DEVELOPMENT

• Quality of work life

• Personal productivity

• Psychological and social well being

• Turnover

• Cultural change

FIGURE B-1 Building evaluation measures using the BSC approach.

APPENDIX B 81

ferent categories of outcomes. Whereas Ouye’s “work-place performance” process focuses on strategicperformance, group performance, and workplace effec-tiveness, this appendix links evaluation more directlyto the four dimensions of the Balanced Scorecard.

ADVANTAGES OF USING THE BALANCEDSCORECARD APPROACH

At the present time, POEs focus on the humanresource dimension of the BSC and primarily on qualityof work life (which includes environmental satisfac-tion, comfort, functional effectiveness of space, accessto resources, etc.) This is an important component, butonly one indicator of the success of the facility. In fact,it is possible for a facility to rate very high on thesecharacteristics, but to have a negative impact on theother areas if the building costs considerably more tooperate and maintain or if the design interferes withkey work processes in some way. This may happen if adesign emphasizes visual openness to enhance com-munication at the expense of ability to concentrate(Brill and Weidemann, 1999).

The BSC approach, in contrast, begins by askingthese kinds of questions:

• How can workplace design positively influenceoutcomes that organizations value?

• How can it reduce costs or increase income?• How can it enhance human resource development?• How can the workplace enhance work processes

and reduce time to market?• How can the work environment enhance customer

relationships and present a more positive face tothe public?

By asking these questions at the beginning of thedesign project, the BSC approach provides an analyti-cal structure to the entire process, from conceptualiza-tion through evaluation and finally to “lessons learned.”For an organization or design firm, these lessonslearned become the knowledge base for future designefforts.

One of the trade-offs inherent in using the BSCapproach, however, is the difficulty of generalizing todifferent contexts. Because the evaluation methodologyis so closely linked to a unit’s own mission and objec-tives, it is difficult to generalize findings to other spacesand units. To deal with this difficulty, a core set ofmeasures could be used across facilities to gain a better

understanding of the entire building stock, while othermeasures would be unique to the goals and objectivesof the particular organization, department, or division(Ouye, 1998).

APPLYING THE BSC APPROACH

In applying the BSC approach, the following gen-eral steps need to be taken:

1. Identify overall organization mission and specificobjectives for each of the four BSC dimensions.

2. Identify how the facility design is expected to helpmeet each objective.

3. Select specific measures for each of the organiza-tional objectives based on links to the workplacedesign. Set performance goals for each measure.

4. Conduct evaluation “pre” and “post.”5. Interpret findings in light of the mission and

objectives.6. Identify key lessons learned.

An example will help illustrate this process.

1. Identify Mission and Objectives

Mission: Become a showcase government office of thefuture.

Specific objectives for each BSC dimension:

• Financial—reduce the costs of modifying facilities.• Business process—reduce time for delivery of

products; create more collaborative working rela-tionships within and between units

• Stakeholder relationships—upgrade the image ofgovernment workspace, increase customer satis-faction.

• Human resources—improve overall quality ofwork life; reduce turnover and absenteeism.

2. Identify Potential Links to thePhysical Environment

This step of the evaluation process is the mostneglected in facility evaluation. It requires consciousarticulation of design hypotheses and assumptionsabout expected links between the specific objectivesand the features of the environment. By making thesepotential links more explicit, it will be easier to inter-


pret results and to assess differences between spacesthat vary on key physical features and attributes.Furthermore, by testing specific design hypotheses, theBSC approach can be used to test and develop theory.In contrast, most POE research is theoretically weakand does not contribute to either hypothesis testing ortheory development.

This step requires research on what is known alreadyas well as logical speculations. The hypothesized linksform the basis for characterizing the baseline and newenvironments. It includes physical measures of theambient environment (thermal, lighting, acoustics, andair quality) and characterization of other key featuresand attributes of the environment that are known orsuspected to influence the outcomes of interest to thestudy. The specific features and attributes used in thecharacterization profiles are related to the objectivesand measures. For instance, if one of the objectives isto reduce absenteeism, then features of the environ-ment known to influence illness need to be cataloguedpre and post. These include materials selection for car-peting and furnishings; ventilation rates; ventilationdistribution; thermal conditions; cleaning procedures;and heating, ventilation, and air conditioning (HVAC)maintenance (Fisk and Rosenfeld, 1997; Heerwagen,2000). The profiles will enable the evaluation team tounderstand better what works and what does not andwhy. They can also be incorporated into a database thatcan be integrated across facilities.

For example, one of the measures cited above forimproved work process is increased collaboration bothwithin and between units, especially support for spon-taneous interactions and meetings. A quick review ofresearch literature provides the basis for developing thepotential links to the environment. Specific questionsto address in the literature review include: How dopeople use the environment for social interactions?What aspects of the environment encourage differentkinds of interactions? How do groups work, and whatresources do they typically use? How important arespontaneous meetings compared to planned meetings,and how do they differ from one another?

Recent research on informal communications andinteractions in work settings shows specific featuresand attributes of informal spaces that are likely to influ-ence the extent to which the spaces are used and theirdegree of usefulness to work groups. These featuresinclude comfortable seating to encourage lingering,location in areas adjacent to private workspaces toencourage casual teaming, white boards for discus-

sions, presence of food nearby, some degree of separa-tion from main traffic routes, acoustical separation fornearby workers who are not participating, ability ofteam members to personalize the space, and ability tomaintain information displays, group artifacts, andwork in progress until these items are no longer needed(Allen, 1977; Sims et al., 1998; Brager et al., 2000).Once these potential features are identified, the baselineand new environments can be assessed to identify theextent to which these features are present.

A similar process would be carried out for each ofthe objectives. For many of the objectives, there islikely to be little research available. Nonetheless, theassumptions, hypotheses, and predictions should stillbe articulated and linkages to the environment shouldbe logically consistent. As another example, reducedtime to market could be influenced by factors such asco-location of people working on the task, easy accessto electronic groupware tools to coordinate work, suf-ficient group space for spontaneous meetings, verticalsurfaces for continual visual display of work in progressand schedules, and central storage for materials associ-ated with the task while it is ongoing.

3. Identify Specific Measures for Each Objective

Many different kinds of measures are likely to beused in the evaluation process. Nonetheless, eachpotential measure should be assessed against generalcriteria to help decide whether or not it should beincluded in the evaluation process. These criteria follow:

• Usefulness—the measure addresses the mission,goals, and objectives of the business unit and canbe used in strategic planning.

• Reliability—the measure produces consistentresults when applied again.

• Validity—the measure is a good indicator of theoutcome of interest (it measures what it purportsto measure).

• Efficiency—the overall measurement plan usesthe minimal set of measures needed to do the joband enables conclusions to be drawn from theentire data set.

• Discrimination—the measures will allow smallchanges to be noticed.

• Balance—the entire set of measures will includeboth quantitative and qualitative measures anddirect and indirect measures. Quantitative datacan be translated into numbers and used for

APPENDIX B 83

statistical analyses. Qualitative data, on the otherhand, often include interviews and results fromfocus groups that are more difficult to translateinto numeric scales. Nonetheless, such data pro-vide a rich understanding of the context and pro-cesses that make it easier to interpret quantitativeresults. Further, qualitative approaches are oftenused as a means to develop items for surveys andstructured interviews or other data-gatheringmechanisms. The second aspect of a balancedfamily of measures is direct versus indirect mea-sures of performance. Direct measures includeoutputs such as sales volume. Indirect measuresare often correlated with performance or are thebuilding blocks of performance rather than actualperformance output. Examples are frequency ofuse, occupant satisfaction, or absenteeism.

Setting Performance Goals. The organization needsto decide prior to the evaluation what degree ofimprovement it is working toward for each of the iden-tified measures. Does even the slightest increase in theexpected outcomes matter? Alternatively, should youaim for a 10 percent improvement, a 25 percentimprovement? Setting performance guidelines willhelp in data interpretation and conclusions. Scientificresearch uses statistical significance as proof ofsuccess. However, statistical analyses may not be asuseful in an applied context. The degree and directionof change over time may be more relevant to organiza-tional performance. Very few performance evaluationprocesses, including the Balanced Scorecard, usestatistical analyses to judge whether organizationalchanges are “working.” Instead, managers look at theoverall profile of outcomes and make a decision aboutnew policies or procedures based on how well the datamatch improvement goals (Kaplan and Norton, 1996).

The following examples identify some potentialmeasures for each of the stated objectives in our hypo-thetical example.

Costs of Modifying Facilities. This would involveidentification and calculation of all costs involved inrelocating workers or reconfiguring office space,including costs associated with packing or unpacking,time and costs of facilities staff needed to reconfigurework stations, time associated with planning the moveand reporting, costs for any special equipment or ser-vices, and lost work time. The data would include totalcosts of staff time (number of hours × hourly pay rate),

total costs of special equipment or services, and theamount of time overall to carry out the change from theinitial request to completion. In order to demonstratereduced costs (the objective), pre and post comparisonswould focus on the total number of people involved ina move and the costs of their time; the overall timeneeded to carry out the change; and the total dollar costsof the move.

Delivery Time for Products. This would requiretracking the time to actually produce a product such asa report, starting with the initial assignment and endingwith the date of final delivery. To identify where in theoverall process the efficiencies occurred, subtaskswould also be timed. Data would include a written com-mentary on work process, number of people involvedin the task and their responsibilities, and reasons forany unusual delays or work stoppages. The best tasksfor such an analysis are reports or other products thatare done on a repetitive basis and therefore are likely tobe very similar from year to year. If different types ofproducts are selected pre and post, then any differencesin delivery times could be due to factors such as taskcomplexity rather than to increased efficiencies result-ing from changes in physical space.

Facility Image. Data on image would include sub-jective assessments through brief surveys completedby visitors, customers, job applicants, and staff. Spe-cific questions would depend upon the nature of thefacility and the type of work. Pre-post analysis wouldlook for changes in perceptions.

Customer Satisfaction. Data on customer satisfactioncould include surveys, analysis of unsolicited customermessages (complaints, concerns, praise), customerretention, and number of new customers. Objectivedata could include the time needed to complete trans-actions with different customers or stakeholders or thenumber of requests for information processed per day.

Inter- and Intra-unit Collaboration. Communica-tions and collaboration activities are notoriously diffi-cult to document accurately unless logs are kept of allmeetings, formal and informal. Furthermore, the valueof collaboration is reflected not only in the frequencyof the meetings, but also in the outcomes of the inter-actions (e.g., new ideas, problems solved more quickly).To get as accurate a picture as possible of changes inmeeting characteristics, multiple methods should be


used. First, motion sensors can be used to gather dataon frequency of use for particular spaces. The sensorswould have to contain counters or other data-processingtechnology (e.g., a sensor that would measure how longthe lights were on) that would ignore short-durationchanges (e.g., someone walking into the room brieflyand then leaving). The occupancy data would have tobe supplemented either with behavioral observationsor surveys and interviews that gathered information onnumber and character of meetings attended within thepast week (or some other limited time period to enhancethe potential for accurate recall). The survey-interviewprocess would also gather data on the attendees, thenature of the meeting (spontaneous versus planned;focused on a specific problem, brainstorming, taskintegration, information exchange, and so forth), andthe perceived value of the meeting (specific outcomes,usefulness, etc.).

Data analysis would compare the number of meet-ings, the participants (number from within the unit,number from other units), the purpose, the outcome,and the perceived value. If the facility had an impact oncollaboration, one would expect to find a wider rangeof participants, more meetings for problem-solving andbrainstorming versus simple information exchange,more spontaneous meetings, and a higher perceivedvalue.

Quality of Work Life. POEs traditionally focus onquality-of-work-life issues such as comfort, environ-mental satisfaction, work experiences and perceptions,sense of place, and sense of belonging. Many designfirms and research organizations have examples ofsurveys that are used in a pre-post analysis.

Turnover. There is a great interest in retainingworkers due to the high costs of turnover, in terms ofboth the financial costs associated with hiring someonenew and the knowledge costs that result from losingvaluable skills and knowledge when a worker leaves.Turnover is usually calculated as a rate of workers whovoluntarily leave an organization divided by the totalnumber of workers for the same time period. Turnoverdoes not include retirements, dismissals, deaths, or lossof staff due to disabling illness. Some degree of staffturnover is important because it introduces new ideasand new skills into an organization. Thus, for evalua-tion purposes, the organization needs to decide whatdegree of turnover is desirable. Pre-post comparisons

of turnover rates should then be assessed against whatis considered a desirable level.

4. Conduct Evaluation Pre and Post

Key issues in conducting the pre-post processinclude gaining cooperation from managers and staffwho will be the study subjects, timing of the evalua-tion, and use of control or comparison groups.

Gaining Cooperation. The evaluation process willfail if the occupants are reluctant to participate or ifthere are insufficient staff to help with the organiza-tional data gathering for some of the measures (such asturnover rates). Occupants are much more likely to con-tinue to be engaged in the process if they are involvedin helping design the measurement plan and if they seea benefit from participation. Having support from high-level organizational leaders is also critical because itsignals the importance of the project. The facility occu-pants also need to be informed of how the data will beused and they need to be assured that their own inputwill be kept confidential.

Control Groups. Because so many other factors caninfluence the outcomes being studied, it is difficult toknow whether performance changes are due to theworkplace itself or to other factors that may changesimultaneously. This is especially true when the designis part of an organizational change effort, which is oftenthe case. Confounding factors may be internal to theorganization (changes in policies or markets), or theymay be external to the organization but nonetheless canaffect business performance (such as economic condi-tions). The best way to avoid problems of interpretingthe success of a design is to use control groups alongwith pre-post studies. An appropriate control groupwould be a business unit in the same building that doesa similar kind of work but is not going through a work-place change. The control group should be as similar tothe design change group as possible.

The control group is studied at the same time as thegroup experiencing the design changes, with bothgroups studied during the pre and post design phases.Although the control group would not experience thedesign change, it would get the same surveys or othermeasures at the same time. If the design has an impactindependent of organizational issues, then the pre-postresponses for those in the design change condition

APPENDIX B 85

should show greater differences across time than forthose in the control group.

Timing of Measurement. The pre-measurementshould be done at least two to four months prior to themove into a new facility in order to avoid issues andproblems associated with the move itself. Ideally, theexisting facility should be evaluated before workbegins on the new facility, although this is very seldomdone due to the need to assemble a research team anddevelop a measurement plan. The post-measuresshould be done six to nine months after project comple-tion to enable workers to adapt to the new setting. Thedelay will help to diminish the “settling-in” phase whenproblems may be most obvious and the workplaceneeds to be fine-tuned. It will also reduce the impact ofa “halo” effect associated with being in a new or reno-vated space.

5. Interpreting Results

When data analysis is complete, the project recon-siders the design hypotheses and asks: Do the datasupport the hypotheses? Do the results meet the perfor-mance goals?

There are very few scientific research studies thatshow complete support for all hypotheses and predic-tions. Thus, we would not expect to find perfect align-ment in design evaluation. Where misalignments occur,it is important to try to understand why this happened.

The design and evaluation teams will have a naturaltendency to focus on the positive and ignore the resultsthat do not turn out the way they expected. However, itis often more valuable to understand why things wentwrong for several reasons. First, you do not want torepeat the mistakes. Second, negative results oftenforce a rethinking of basic assumptions and a searchfor better links between the environment and thebehavioral outcomes.

A problem with all facility evaluations, regardlessof specific methodologies, is the issue of causation. If anew facility is found to have positive outcomes, canthese be attributed unequivocally to the physical envi-ronment and not to other factors? The answer is clearlyno. Causation can be determined only by carefullydesigned experiments that vary only one component ata time. Since this is unrealistic in field settings, thecausation issue will always be present. The best thatcan be done is to minimize other explanations to thedegree possible through the use of control and com-

parison groups. It will also be important to use a highdegree of logic in interpreting results, to look for con-sistency across facilities that share similar features, andto look at relationships between measures.

For instance, if absenteeism is of interest, thenabsenteeism rates should be associated with other out-comes, such as symptom expression or low levels ofmotivation, both of which could lead to taking days offdue to illness or lack of desire to come to work. Assess-ing patterns of absenteeism will further aid in interpre-tation of results. Absence associated with motivationalissues is likely to have a different pattern of days offthan absenteeism due to illness. Because illnesseshappen randomly and often last for more than one day,absenteeism due to illness should be clustered andrandomly distributed over the days of the week. Moti-vational absenteeism, on the other hand, is more likelyto occur on particular days of the week (especiallyMonday or Friday) and would be more likely to occurjust one day at a time, not for several days. Anotherway to assess absenteeism is to look at its opposite—attendance. Attendance can be assessed by determin-ing the percentage of workers with perfect attendanceor the percentage who used less than the allowednumber of sick days in the year prior to and the yearafter the move to the new facility. In addition to look-ing at relationships between measured outcomes, thereshould also be a logical connection to the physical envi-ronment profiles. Absenteeism and illness symptomsshould be associated with factors such as poor indoorair quality and low maintenance of HVAC systems.

Another problem for interpreting the results onfacility evaluation is that redesign often goes hand inhand with organizational change. Thus, positive (ornegative) results could be due to organizational issuesand not to the physical environment. This is where con-trol groups become very valuable. If the organizationalchange is widespread, then similar units should alsoexperience the effects of these changes. Thus, differ-ences between the control group and the group in thenew space are more likely to be related to the environ-ment. Again, the use of logical thinking is also impor-tant. When organizational change occurs, some aspectsof behavior are more likely to be influenced than others.For instance, if staff perceive the change very nega-tively, then motivationally influenced absenteeism maygo up in both the new and the control spaces. Otheroutcomes, such as the costs of “churn,” are less likelyto be affected by organizational change policies.

Greater assurance of a true connection between the


physical features of the space and the measured out-comes can be gained also by using a geographicalinformation system (GIS) approach to data analysis.Outcomes on various subjective measures can beplotted on floor plans to gain a greater understandingof the spatial distribution of responses. For instance, aGIS format used by the author to assess environmentalsatisfaction and comfort in an office building in Cali-fornia clearly showed that problems associated withdistractions occurred primarily in particular locations,regardless of whether people were in private offices orcubicles. Similarly, thermal and air quality discomforttended to cluster more in some areas than others. At thepresent time, most post-occupancy data analysis useshuman characteristics as the primary unit of differen-tiation (e.g., different job categories, gender, age), withcomparisons in responses across job categories or age.By supplementing the demographic data with geo-graphical analysis, the evaluation will provide a morecomplete picture of the facility. A similar process,called spatial modeling, has been suggested by Aronoffand Kaplan (1995). Both the GIS and the spatial model-ing approaches allow for analysis of the variability anddistribution of responses in a spatial format.

7. Identifying Lessons Learned

An issue with lessons learned is: Where should theknowledge reside—in people’s heads or in the envi-ronment? Should the lessons be internalized andbecome part of one’s tacit operating knowledge, orshould the lessons be located for anyone to access—inreports, databases, and so forth? Both should happen.If people are going to work with the knowledge gained,they need to incorporate it into their everyday ways ofthinking and working. Internalization takes time andcontinued work with the knowledge and issues (Norman,1993; Stewart, 1999). Once internalized, knowledge ispart of a person’s intellectual capital and leaves theorganization when the person does. This is why knowl-edge also needs to be made explicit so it can become anorganizational asset, not just a personal asset (Stewart,1999). Seminars and presentations on the results offacility evaluations—with both positive and negativeresults highlighted—should be an ongoing practice.

Since a major purpose of evaluation is to apply theknowledge gained to future projects, simple databasesthat could be accessed by key words would be espe-cially valuable to future designs. The database shouldinclude the design hypotheses and assumptions for each

project, the specific measures used to test the hypothesis,pre and post photos of the space, brief summaries ofthe data, key lessons learned, connections to otherstudies, connections to the full research findings preand post, and recommendations for future designs. Thepresentation of lessons learned should be as visual aspossible to allow for maximum understanding andretention (Norman, 1993). Graphs, photos, and keywords and concepts are much more likely to be usefulthan long verbal explanations that can be accessed ifdesired through links to full reports.

In addition, simple methods to display overall resultswould aid interpretation and lessons learned. Forinstance, results could be visualized using color-codedicons to provide an easy visual interpretation: greencould be used to show strong support for the hypothesisand meeting or exceeding performance goals; yellowcould be used to show minimal support or no change;and red could be used for measures that did not supportthe hypotheses or showed negative change.

SUMMARY AND CONCLUSIONS

This appendix has described an approach to post-occupancy evaluation that is more closely linked tobusiness and workplace strategies than existing meth-odologies. Although the Balanced Scorecard approachdoes not present any new measurement techniques orbreakthrough methodologies, it does provide a processfor more effectively linking facilities to an organiza-tion’s overall mission and goals. An advantage to usingthe BSC approach for federal facilities is its close rela-tionship to the comprehensive performance assessmentrequired by GPRA. Traditional POEs provide animportant source of input, but measures tend to befocused on a limited range of topics and on the occu-pants’ perspective, rather than on the broader, strategicfocus of the BSC.

For large real estate portfolios, such as those in fed-eral departments, the determinant of facility success isnot only how well the overall building stock performswith respect to core POE measures used across facili-ties, but also how well each design fits its particularcontext and how well it meets the business objectivesof the unit. The Balanced Scorecard was developedspecifically for the purpose of providing data to assessoverall performance and to identify areas that needattention.

APPENDIX B 87

ABOUT THE AUTHOR

Judith H. Heerwagen is an environmental psycholo-gist whose research and writing have focused on thehuman factors of sustainable design and workplaceecology. Dr. Heerwagen currently has her own con-sulting and research practice in Seattle. Prior to start-ing her own business, Dr. Heerwagen was a principalwith Space, LLC, a strategic planning and design firm,and a senior scientist at the Pacific Northwest NationalLaboratory. At Space Dr. Heerwagen was codirector ofresearch and helped develop metrics for the WorkplacePerformance Diagnostic Tool. At the Pacific NorthwestNational Laboratory she was responsible for developingresearch methodologies to assess the human and orga-nizational impacts of building design. Dr. Heerwagenhas been an invited participant at a number of nationalmeetings focused on workplace productivity. She wason the research faculty at the University of Washingtonin the College of Architecture and Urban Planning andat the College of Nursing. Dr. Heerwagen is a memberof the American Psychological Association. She holdsa bachelor of science in communications from the Uni-versity of Illinois, Champaign-Urbana, and a Ph.D. inpsychology from the University of Washington.

REFERENCESAllen, T. (1977). Managing the Flow of Technology. Cambridge, Mass.:

MIT Press.

Aronoff, S., and Kaplan, A. (1995). Total Workplace Performance:Rethinking the Office Environment. Ottawa: WDL Publications.

Becker, F., and Steele, F. (1995). Workplace by Design: Mapping the HighPerformance Workscape. San Francisco, Calif.: Jossey-Bass.

Brager, G., Heerwagen, J., Bauman, F., Huizenga, C., Powell, K., Ruland,A. and Ring, E. (2000). Team Spaces and Collaboration: Links to theEnvironment. Berkeley: University of California, Center for the BuiltEnvironment.

Brill, M., and Weidemann, S. (1999). Workshop presented at theAlt.Office99 Conference, San Francisco, Calif. December.

Fisk, W., and Rosenfeld, A.H. (1997). Estimates of improved productivityand health from better indoor environments. Indoor Air 7:158-172.

General Services Administration. (1999). The Integrated Workplace: AComprehensive Approach to Developing Workspace. Washington, D.C.:Office of Governmentwide Policy and Office of Real Property.

Grantham, C. (2000). The Future of Work: The Promise of the New DigitalWork Society. New York: McGraw-Hill, Commerce Net Press.

Heerwagen, J. (2000). Green buildings, organizational success and occu-pant productivity. Building Research and Information 28(5/6):353-367.

Horgen, T.H., Joroff, M.L., Porter, W.L., and Schon, D.A. (1999). Excel-lence by Design: Transforming Workplace and Work Practice. NewYork: Wiley.

Kaplan, R.S., and Norton, D.P. (1996). The Balanced Scorecard. Boston:Harvard Business School Press.

Norman, D. (1993). Things That Make Us Smart: Defending HumanAttributes in the Age of the Machine. Reading, Mass.: Addison-Wesley.

Ouye, J.O. (1998). Measuring workplace performance: Or, yes, Virginia,you can measure workplace performance. Paper presented at the AIAConference on Highly Effective Facilities, Cincinnati, Ohio, March12-14.

Ouye, J.O., and Bellas, J. (1999). The Competitive Workplace. Tokyo:Kokuyo (fully translated in English and Japanese).

Sims, W.R., Joroff, M., and Becker, F. (1998). Teamspace Strategies:Creating and Managing Environments to Support High PerformanceTeamwork. Atlanta: IDRC Foundation.

Stewart, T.A. (1999). Intellectual Capital. New York: Doubleday.

Appendix C

Supplemental Information to Chapter 3

APPENDIX C 89

Buildings-in-Use

1

Buildings-In-Use Boston Montreal

December 20, 2001

I.D.Number ____/___/____

Please leave blank

Welcome to the Building-In-Use Assessment Survey!

This questionnaire is for all staff. We want to find out more about how you feel

about the facility you work in, and how you feel this environment affects your

work.

Below you will find a checklist of items about your workspace. Please answer

these questions as soon as you receive the questionnaire. It will take you less

than 10 minutes to complete. When you have filled it out, please return it

immediately.

Please do not fill out the ID number on this survey form. It is used for analysis

purposes. However, please provide your office location in the space provided,

as this will help us understand the building conditions at your work location.

Your name is not necessary on the questionnaire and your answers will remain

confidential.

We really want to hear from you. Thank-you for participating!

PLEASE FILL OUT THE FOLLOWING: Office or cube number _________________

Floor _________ Workgroup or department name _______________________

Please rate your comfort level in your primary workspace on the following

scales, where 1 is poor or uncomfortable and 5 is good or comfortable, and 2 3 -

4 are in-between, with 3 being neutral. Your task is to circle the number on each

scale that best represents your experience of working in this building.

1. Temperature comfort:

1 2 3 4 5

GENERALLY BAD GENERALLY GOOD

over . . .

271 Lincoln Street

Lexington

MA.02421

781-674-3186 phone

781-674-1489 fax


Buildings-in-Use

2

2. How cold it gets:

1 2 3 4 5

TOO COLD COMFORTABLE

3. How warm it gets:

1 2 3 4 5

TOO WARM COMFORTABLE

4. Temperature shifts:

1 2 3 4 5

TOO FREQUENT GENERALLY CONSTANT

5. Ventilation comfort:

1 2 3 4 5

GENERALLY BAD GENERALLY GOOD

6. Air freshness:

1 2 3 4 5

STALE AIR FRESH AIR

7. Air Movement:

1 2 3 4 5

STUFFY CIRCULATING

8. Noise distractions:

1 2 3 4 5

DISTURBING NOT A PROBLEM

9. General office noise level

(background noise from

conversation and equipment):

1 2 3 4 5

TOO NOISY COMFORTABLE

10. Specific office noises

(individual voices

and equipment):

1 2 3 4 5


over . . .

APPENDIX C 91

Buildings-in-Use

3

11. Noise from the air systems:

1 2 3 4 5


12. Noise from office lighting:

1 2 3 4 5

BUZZ/NOISY NOT A PROBLEM

13. Noise from outside the building:

1 2 3 4 5


14. Furniture arrangement in your workspace:

1 2 3 4 5

UNCOMFORTABLE COMFORTABLE

15. Amount of space in your workspace:

1 2 3 4 5

INSUFFICIENT ADEQUATE

16. Work storage:

1 2 3 4 5


17. Shared (team) file storage:

1 2 3 4 5


18. Personal storage:

1 2 3 4 5


19. Visual privacy in your workspace:

1 2 3 4 5


over . . .


Buildings-in-Use

4

20. Voice privacy in your workspace:

1 2 3 4 5


21. Telephone privacy in your workspace:

1 2 3 4 5


22. Electrical Lighting:

1 2 3 4 5


23. How bright lights are:

1 2 3 4 5

TOO MUCH LIGHT DOES NOT GET TOO BRIGHT

24. Glare from lights or windows:

1 2 3 4 5


25. Natural lighting from windows:

1 2 3 4 5

INSUFFICIENT LIGHT GOOD NATURAL LIGHT

26. Not enough light:

1 2 3 4 5

TOO DARK COMFORTABLE

27. Please rate how this space affects your ability to do your work:

1 2 3 4 5

MAKES IT DIFFICULT MAKES IT EASY

28. How would you rate your satisfaction with this building?

1 2 3 4 5

DISSATISFIED VERY SATISFIED

over . . .

APPENDIX C 93

Buildings-in-Use

5

PLEASE COMMENT:

29. What I like best/find most useful about this building as a place to work :

30. What I dislike most/have most trouble with in this building as a place to work :

Appendix D



APPENDIX D 97


APPENDIX D 99


APPENDIX D 101


APPENDIX D 103


APPENDIX D 105


APPENDIX D 107


APPENDIX D 109


APPENDIX D 111


APPENDIX D 113


APPENDIX D 115

116

Appendix E


COMPANIES OFFERING ONLINE SURVEYS AND/OR POLLING SERVICES1

In general, each package will have the following fea-tures, to some level of ease:

• installation and integration of software into asystem

• supporting documentation, such as online help,tutorial, or a guide on how to make a survey

• the ability to add, edit, and manage templatesprovided

• options for building single- and multiple-pageforms and branching to other questions within orbetween pages

• ability to scan e-mail or data files as the resultscome in

• file management features such as importing andexporting data, data cleaning, and record keeping

• the ability to post surveys on the Web and pro-vide support to a server

• data analysis tools and types of analysis available• options to chart and present data• overall ease of using the product and its user in-

terface

THE CHANGING CONTEXT OFONLINE COMMUNICATIONS

Today, anyone with a cyber address is inundatedwith unsolicited messages and unnecessary communi-cations, often originating from within their own orga-nizations. The exponential growth of junk e-mail in

recent years is a phenomenon termed “spam” (noxious,unwanted e-mails). Using current communicationstechnology, a single cyber-marketing company cansend half a billion personalized ad mails via the webevery day. It is estimated that it costs Internet usersworldwide $US 9-billion ($CDN 14-billion) annuallyto receive junk e-mails (Hargreaves, 2001). In this envi-ronment, people may not bother to open unsolicitede-mail or to agree that a survey be sent to them.

The low response rate for online surveys might alsoreflect a general mistrust of electronic communication.For example, unbeknownst to users, their consumerinformation may be gleaned while they visit Web sites.Then this information can be sold for large sums ofmoney and so it escalates. Having been damaged bytempting messages, such as the “I love you” virus, com-puter users may now be more cautious of electronicinvitations, limiting their willingness to participate inonline surveys. This would apply to wide-cast cyber-

TABLE E-1 Companies Offering On-line Surveys orPolling Services.

EZSurvey 2000 SurveySolutions forwww.raosoft.com the Web 3.0

www.perseus.com

SurveyCrafter Professional 2.7 WebSurveyor(previously MarketSight 2.5) www.websurveyor.comwww.surveycrafter.com

Survey Select 2.1 and ZoomerangSurvey Select Expert 4.0 www.zoomerang.comwww.surveyselect.com

1See King (2000) for reviews of the software.

APPENDIX E 117

surveys, and less so to e-surveys sent through anorganization’s proprietary network/intranet.

There are Web users who pay for their time online.That could deter some from spending valuable minutesto fill out a survey. Eliminating these potential respon-dents both lowers the response rate and might also adda bias based on income.

On the other hand, the cost to connect is steadily com-ing down and there are increasing opportunities for thegeneral public to access the Internet. Businesses such aseasyEverything <www.easyeverything.com>, Kinko’s<www.kinkos.com>, and Get2net <www.get2net.com>are filling storefronts in city centers. At easyEverythingin Manhattan there are 800 terminals with Internetaccess and one dollar ($US1) buys two hours of con-nectivity. According to Pike (2001), there is an inter-esting cast of characters accessing the net at 11PM onSaturday night at the Times Square location. Kinko’soffers a fast connection to surf the Internet and useMicrosoft’s complete Office Suite (Word, Excel,PowerPoint) for thirty cents a minute. Get2net has freeInternet kiosks at select locations, however keyboardsare awkward, access slow, and there’s lots of advertising.

DETAILS OF WHO IS ONLINE AND WHERE THEYARE GEOGRAPHICALLY

The number of people accessing the Internet contin-ues to increase at a phenomenal rate. In 1995 TheInternet Society estimated that between 20 to 40 mil-lion people around the world had access to the Internet.Nua Internet Surveys (Nua, 2001) estimated that num-ber to have grown to 201 million worldwide in 1999,and up to 407 million by 2000. See Table E-2.

Early Internet users (circa 1995) tended to be young,white males with high socioeconomic status. Recentstudies suggest that as more people use the Internet andWorld Wide Web, there is a demographic shift and thatInternet users are beginning to represent more of thegeneral population. More households have Internetconnections. The US Department of Commerce (1999)reported that the number of households connected tothe Internet increased from 18.6% in 1997, to 26.2% in1998.

The take-up of electronic communications is fasterthan any other “disruptive technology” of the 20th cen-tury—namely electricity, the telephone, and the car. Ingeneral, a medium is considered a “mass medium”when a critical mass of people (about 16% of the popu-lation, or 50 million for the USA) has adopted the inno-

vation (Markus, 1990). It took 38 years for radio toreach this level of adoption. Television took 13 yearsand cable television reached a critical mass in 10 years.Depending on the various estimates on the number ofInternet users, the medium has already reached criticalmass or will certainly be there by 2002, just 8 yearsafter its emergence as a consumer medium (Neufeld,1997).

Although a large number of people access the Web,in 1998 they accounted for less than one third of theoverall USA population (Kaye and Johnson, 1999).Estimates vary, and as much as half the USA popula-tion may be connected. The fast take-up of this mediumis rapidly changing the profile of who’s online, makingless relevant some of the lessons-learned and samplingissues from earlier work. The trends suggest that thenumber of users will continue to grow, will betterreflect the overall population, and that upwards of 80%of Internet users will access the system daily. Such auser base would provide a reliable population fromwhich to sample and generalize findings.

REFERENCESHargreaves, D. 2001. Junk e-mail costs online surfers $14-billion a year:

EU report. Financial Post. February 3. p. D9.Kaye, B. and T. Johnson. 1999. Research Methodology: Taming the Cyber

Frontier. Social Science Computer Review. Volume 17, No 3, pp.323-337.

King, N. 2000. What are they thinking? PC Magazine. February 8, pp.163-178.

TABLE E-2 Top Ten Countries with Internet Users -Number and Percentage of Users.

Population Internet Users % of PopulationCountry (in million) (in million) on Internet

Australia 19 7.4 38.9%United States 276 91.0 33.0%Canada 31 9.7 31.3%Japan 127 29.0 22.8%Germany 83 18.9 22.8%United Kingdom 60 18.8 31.3%South Korea 46 14.0 30.4%France 59 10.7 18.1%Italy 58 6.6 11.4%China 1,300 10.0 0.8%

Source: Morgan Stanley Dean Witter, 2001.


Markus, L. 1990. Toward a “critical mass” theory of interactive media. InFulk, J. and C. Steinfield (eds.) Organizations and communication tech-nology. pp. 194-218. Newbury Park, CA: Sage.

Morgan Stanley Dean Witter. 2001. In Infoworld. March 12, p. 16.Neufeld, E. 1997. Where are the audiences going? MediaWeek. May 5, pp.

S22-S29.

Nua Internet Surveys. 2001. http://www.nua.ie/surveys/how_many_onlineUS Department of Commerce. 1999. Falling through the net: defining the

digital divide. National Telecommunications and Information Adminis-tration. 27 pages.

119

Appendix F

Chapter 5 from Post-Occupancy Evaluation Practices in theBuilding Process: Opportunities for Improvement,

National Academy Press, 1987

TRENDS, CONCLUSIONS,AND RECOMMEDATIONS

Based on observed trends, the committee makes thefollowing conclusions and recommendations to be con-sidered by government agencies and private organiza-tions responsible for construction programs in generaland for conducting POEs in particular. These recom-mendations recognize the current lack of institutionalsupport for this field, as well as the need to generate areliable and comprehensive data base. To that end, thecommittee proposes measures that will: (1) make POEa more systematic process with rigorous procedures,(2) lay the groundwork for a data base of knowledge onbuilding use and performance, and (3) establish aclearinghouse to assemble, maintain and disseminateinformation generated by POEs.

The committee presents trends, conclusions, recom-mendations and discussions (in that order) in threesections: (1) policy-related topics that focus on broadpolicies that should be instituted to make POE moreuseful and widely used, (2) building process-relatedtopics that focus on procedures in the uses of POE, and(3) technology and techniques-related topics that iden-tify innovative ways in which POEs can be improved.

ITEMS RELATED TO POLICY IN THEBUILDING PROCESS

This section presents three policy-related topics:(1) monitoring building quality and performance,(2) POE data base and clearinghouse information, and(3) POE data and litigation. The committee recom-mends courses of action that can be implemented to

make POE more beneficial to government agencies andprivate organizations, as well as to improve the plan-ning, design, construction and operation of facilities.

Monitoring Building Quality and Performance

Trend 1. Quality assurance programs are used bymany manufacturing concerns to raise consumer confi-dence and to compete more effectively in worldmarkets. POE addresses a significant part of qualityassurance in the building industry. As each facility isevaluated in use, the existing quality of materials anddesign concepts can be critically assessed, and designcriteria can be changed to produce better facilities inthe future.

Hidden high operating costs, costly repairs, and dys-functional facilities have made administrators moreaware of the need for quality buildings. This is espe-cially true for those institutions that build many facili-ties on a recurring basis.

POE also constitutes an “auditing” tool that can beused by the knowledgeable client. Together with con-struction audits and other recognized financial account-ing practices, POE can track performance of a project,document costly changes to the program requirements,and identify critical strengths or weaknesses associatedwith a particular facility type. POE can be used as aparallel track in the design and construction process,tracking decisions, changes, and outcomes.

Conclusion 1. More emphasis is being placed onquality in our society today. Occupants of facilitiesexpect that same quality in terms of building perfor-mance. Organizations in the private and public sectors


are concerned about the price/performance relationshipfor new facilities and, therefore, want to developresponsive buildings for the lowest possible cost.

Recommendation 1. Government agencies manageand operate a significant real estate portfolio for theirown account. This includes offices, hospitals, housingand special use facilities. Agencies should be learningand benefiting more from their extensive design, con-struction and operational experience. Through POE,they should be applying the lessons learned to reduceoperating costs, to design environments that improveproductivity, and to build facilities that respond to therapidly changing requirements of institutional users andclients.

Discussion 1. By evaluating the performance of newand existing facilities in terms of how well they workfor the user, agencies can make trade-offs on futureprojects and target features that have the greatest returnin assuring building quality and performance. Further-more, professionals in the design, construction andfacilities management community can exchange infor-mation with one another through associations, con-ferences and written presentations. By exchanginginformation generated by POEs about buildings in use,they will greatly expand knowledge about how toachieve better quality buildings.

Development of Data Bases and Clearinghouses

Trend 2. Today there is the technology and capabilityto develop electronic data bases for use by various par-ticipants in the building industry. Some large corpora-tions have already begun developing these data basesto disseminate information to subscribers and other in-terested users. Certain industries have establishedclearinghouses to collect, organize, archive and dis-seminate specialized information. Clearinghouses havehelped to advance practice in those fields that buildupon precedent and other professionals’ work. In thebuilding industry, several professional groups (such asthe International Facilities Management Associationand the Building Owners and Managers AssociationExchange) are involved in establishing clearinghouseactivities. At this time, however, they are not designedto handle POE information.

Conclusion 2. Because the operation of facilities isbecoming increasingly complex, the sharing of knowl-

edge and experience takes on added importance. POEresults can be organized into a data base format and canbe made available to subscribers through a clearinghouse,subscription service, or an electronic data base. Stan-dardized documentation and data collection can beused, and government building projects can be enteredand evaluated as part of the data base. The clearing-house would manage the information and provide acentral source of expertise.

Recommendation 2. Government agencies, togetherwith private sector organizations, should create andsupport: (1) an on-line data base that would containPOE results, design criteria, and other design guide-lines, (2) a clearinghouse consisting of electronic databases, and case studies, and (3) POE networks, directo-ries, conferences, and other ways within governmentagencies to expedite the exchange of knowledge.

Discussion 2. Various academic and research-orientedassociations already have extensive, informal POE net-works. Currently, these networks are developed andmaintained through voluntary efforts. Federal agencies,at very little expense, could put together a directory ofindividuals and groups who conduct POEs or who areinterested in doing so. Various forms of informationexchanges and other supporting materials could beassembled. The data base would be kept updated byincoming POE reports, by special studies related toimportant design concepts, and by other research. Itwould be set up so as to be accessible through nationalnetworks already established.

A centralized capability to organize, collect and dis-seminate POE-based knowledge is critically needed.New building projects are usually begun withoutknowledge of how previous solutions have fared; toooften new design is based on architectural trends,aesthetics or first-cost considerations. The cost to thegovernment is enormous as novel building designs fail,buildings do not satisfy the needs of users, and costoverruns mount because of unanticipated problems.

A clearinghouse, as envisioned by the committee,would primarily use electronic data bases that could beaccessed by private and public sector users on a fee-for-service basis.

POE Data and Litigation

Trend 3. Increased use of litigation in our societyraises a concern about the possible use of POE data in

APPENDIX F 121

lawsuits. The fear is that responsibility will be attrib-uted from POE data to certain parties, and lawsuits andcostly legal expenses may result.

Conclusion 3. The possibility of POE data used inlawsuits may potentially have a crippling effect on thecontinued development of the field. Actions should betaken to safeguard the use of POE results.

Recommendation 3. In conjunction with other policyactions, adequate information controls and safeguardsshould to be developed and implemented in any POEprogram. A legal and ethical code is also required tocover POE use. Public sector agencies, workingthrough the Federal Construction Council, shouldrequest the Building Research Board (or similarorganization) to develop appropriate procedures andsafeguards.

Discussion 3. Safeguards need to be built into anyPOE program to insure accurate reporting, to minimizenuisance suits, and to protect the parties involved in thedesign and construction of buildings. If POE becomesassociated with punitive litigation in this way, agencypersonnel may refuse to do POEs. The results of POEsshould remain confidential until clearance by the clientorganization.

ITEMS RELATED TO PROCEDURES IN THEBUILDING PROCESS

This section considers four topics related to POEpractice and procedures: (1) the building performanceconcept and standards, (2) changing human require-ments and building technologies, (3) user participationand training, and (4) economics. The committeebelieves that POE can significantly improve buildingsby promoting research-based programming and design.The results of POE can be used to identify key factorsabout building performance that make the operationand management of buildings more efficient and costeffective.

The Building Performance Concept and Standards

Trend 4. Higher quality buildings can be developed asPOE data bases come into general use. The results ofPOE will provide designers with an empirical base onthe performance of buildings that can be used to assessother buildings and to evaluate new design concepts.

The results, factored into the building process throughupdated codes and revised criteria, will promote greaterquality design solutions. Unified and accepted stan-dards allow for the communication and comparison ofdata from individual studies. Such standards develop ahigher level of professionalism in the field; prac-titioners adhere to these practices, thus allowing for thecomparison of findings and the interpretation of results.

Conclusion 4. Buildings are designed based on cer-tain goals and performance requirements that are fur-ther clarified by defining explicit, often quantitative,performance criteria and by establishing a range ofmeasured values that will satisfy those criteria. Sincemany POEs in current practice are ad hoc in character,there is little basis for comparisons or for valid infer-ences to be drawn. A systematic POE program wouldenhance the ability of a regulatory agency to verifycompliance with the performance features of its codesand standards. Rigorous POEs would fill this need,increasing design flexibility as well.

Recommendation 4. Key indicators and reliable,objective building performance measures should bedeveloped for use in POEs, and as a basis for designcriteria, as well as standards and guidelines for a varietyof common facility types. In addition, the descriptionand documentation of the buildings being evaluatedshould be improved.

Discussion 4. A set of key indicators, similar to theeconomic indicators used in evaluations of the state ofthe economy, should be used in POEs. These key indi-cators, associated with other, more standardizedmethods and procedures, would lead to more reliableresults that could be more easily communicated. Inaddition, efforts should be made to develop perfor-mance requirements (e.g., purpose, description, assess-ment, conclusions of lessons learned, strengths andweaknesses) that can be used at different levels ofevaluation—from walk-through POE ratings to morein-depth diagnostic POEs.

Several levels of POE investigation can be under-taken. Each of these levels has somewhat differentobjectives and requires a somewhat different set ofmethodologies, procedures, and related formats. Theseprocedures should be standardized so as to allow forthe comparability of information and results. Such stan-dardization, including how to handle exceptions, would


allow data to be entered into a data base and could bemade available to all pertinent government agencies.

A taxonomy of buildings that describes meaningfulcategories of features, materials and systems can bedeveloped to create a common basis for comparisonsand evaluations. The physical environment that is beingevaluated in a POE needs to be adequately described sothat one study of that facility type can be related toanother. A set of descriptor categories as well as a setof physical, objective measures (e.g., lighting levels onthe work surface) should be included.

Changing Human and Building Parameters

Trend 5. Users expect environments that are respon-sive to their needs. Occupants are the critical elementin helping organizations to achieve their mission, andfacilities must support the needs of building occupants.The introduction of new materials and technologiesinto the building industry will continue as producersdevelop new products and applications. This willpresent opportunities for new design strategies andsolutions; it will also present new dangers. Some ofthese products are tested and evaluated in a laboratorysetting, but problems are only identified after anextended period of use (e.g., gases being given off fromsome office products contributing to poor indoor airquality) or when a disaster occurs (e.g., toxic fumesbeing produced by the burning of certain plastic mate-rials in furniture). There is a need for evaluating thesematerials and technologies in use when they are com-bined with other products or put to unusual and noveluses.

Conclusion 5. New technologies, a consumer ethic,and more education are changing the way people usedesigned environments. Environments must becomemore flexible to accommodate frequent changes, andthey must be more responsive to provide for newlyemerging needs. New materials and technologies allowthe designer to create specialized spaces in buildings tohouse a variety of activities.

New materials (e.g., plastics, bonding agents, andsealants) are being developed for use in buildings,building systems and furniture. Similarly, new tech-nologies are being introduced into buildings that allowfor improved building operation or new design options.New computer-based technologies that augment theactivities of building occupants are placing newdemands on building systems and performance.

Recommendation 5. POE programs should be devel-oped that allow facility management to assess and planfor the changing requirements of building occupants.POEs can be used to evaluate existing buildings, regu-larly assess users’ perceptions of the facilities, and planfor necessary changes based on user needs.

POE practices should be employed to evaluate newmaterials and technologies in actual use. Prototypes orrepresentative cases should be identified and studied;the results from evaluations could then be generalizedto future applications.

Discussion 5. Organizations change to meet newconditions; as a result, individuals within organizationsfrequently move or change activities. A POE programthat regularly evaluates facilities from the users’ per-spective can respond to these ongoing changes. Newneeds can be anticipated; facilities can be fine-tuned orretrofitted. Building maintenance priorities can beestablished on safety concerns and occupants’ needs.Facility management can then supply environmentsthat are flexible, respond to the changing needs of users,and are satisfying places in which to live and work.

A POE program, especially one that incorporatesresults into clearinghouses and electronic data bases,can be used to spot potential problems or trends acrossvarious facility types before a disaster occurs or aserious health hazard develops. With an early identifi-cation program, agencies can avoid possible litigationor costly retrofits to resolve health-related problems.Liaisons with other agencies that act on behalf of thepublic welfare or conduct research related to health andsafety problems could be established. Liaisons withproduct testing laboratories, manufacturing associa-tions or professional societies would also promote thesharing of results and rapid communication if potentialproblems were identified. POE programs could pro-vide a testing capability for building products, materi-als and technologies in actual use. These evaluationscould also furnish valuable research data for productmodifications.

User Participation and Training

Trend 6. User participation: Focus groups, userpanels, surveys and other forms of market research areused extensively in consumer product development toestablish user preferences and product acceptance. Userbehavior patterns and human factor considerations havebeen recently added to product development efforts.

APPENDIX F 123

Training: Organizations that develop and managefacilities for their own users are finding it increasinglynecessary to professionalize their in-house staff, whichis expected to be proactive (i.e., anticipate change andplan for change). Such an expectation of professional-ism requires more training of existing in-house staff.

Conclusion 6. User participation: User participationis an accepted practice, which in the building field canboost morale, improve office productivity, and provideother user benefits.

Training: As organizations professionalize theirfacilities management staff, they need to providespecialized training and education programs. Theseprograms should deal with planning and implementingchange, using the environment to support organiza-tional objectives, involving users in the planningprocess, and implementing facilities managementprograms.

Recommendation 6. User participation: POE pro-grams that solicit end user feedback and informationshould be used to heighten participation in the designof new facilities or in improving existing ones.

Training: Certification or other training programsshould be developed to educate agency personnel ortheir consultants regarding concepts and techniques ofconducting POEs.

Discussion 6. User participation: Typically, clientrepresentatives of the building owner make decisionsfor most people in the organization. Often, they do nothave first-hand experience of various functions, nor dothey know the personal preferences of individuals. Itwould be more effective to have end users participateby expressing attitudes, personal preferences, behav-ioral styles, and other characteristics of a more per-sonal nature. This higher level of participation by userswould provide a richer representation of user needsfrom which to develop new design solutions.

Training: Pilot training programs could be createdby knowledgeable POE practitioners to train in-housepersonnel or consultants on how to do POEs. This couldbe done through universities and technical schools.Printed instructional material could be supplementedby videotaped materials of case studies, or tutorialsdocumenting how a POE is conducted. Seminars orworkshops could also be established for designers,facility managers, building operators, and real estateconsultants. Professional associations such as the Inter-

national Facilities Management Association, the Build-ing Owners Management Association, the NationalOffice Products Association, and others intimatelyinvolved with the building industry should be enlistedin this training effort.

Economics

Trend 7. Life-cycle costing, including the costs ofoperation, maintenance, and other facility-relatedactivities, is an increasingly important consideration forinstitutions that develop their own facilities. Facilitymanagers can adopt POE procedures to project thequality of a facility that they build, as well as to evalu-ate its performance over time.

Conclusion 7. Increasingly, organizations are becom-ing more aware of the value of the fixed asset basewhich they own, manage and operate. Facilities man-agement as a professional occupational category hasalso grown, as owners adopt a more active postureregarding the management of their facilities. It isexpected that POE will generate significant costsavings by improving design criteria, by correctingproblems that are discovered after building occupancy,and by improving the overall building stock over thelong term.

Recommendation 7. POEs should become part of themanagement process used by facility managers, build-ing operators, and others responsible for fixed assetmanagement, new facility development, or design.Research on building economics and the overall life-cycle costs associated with a facility should be con-ducted. Research should also be done on the costs ofPOEs, including savings that are realized as a result ofPOEs.

Discussion 7. There has been a growing recognition ofthe economic importance of high quality buildings andtheir management as fixed assets. Similar to the humanresource or information resources management func-tions, facility management is concerned with the day-to-day operations and, occasionally, with new buildingprojects.

The economics of building occupancy are related tothe housing of personnel, technology and various func-tions of an agency and should be viewed from a build-ing life-cycle perspective. Similarly, POE should beseen as part of the overall building process, and not as


singular case studies. To that end, more economicresearch is needed to document savings and oppor-tunities.

As POE is increasingly used to provide the industrywith empirical data about buildings in use, POE resultscan document manufacturers’ claims, give perfor-mance profiles of individual buildings systems, andprovide information about possible trade-offs.

ITEMS RELATED TO INNOVATIVETECHNOLOGIES AND TECHNIQUES

This section reviews four areas of POE practice per-taining to: (1) smart buildings, (2) computer-based sys-tems, (3) simulations, and (4) mathematical modeling.

Smart Buildings

Trend 8. It is now possible to build into a facility thecapability to monitor constantly or frequently occu-pants’ responses. Individuals are able to provide feed-back to facility managers through interactive, comput-erized “check-out” procedures. It may soon becomepossible to develop electronically monitored environ-ments that respond to commands of building occupants.There will also be the capability of providing electroni-cally adjusting environments that automatically changeto meet the needs of the occupants.

Conclusion 8. With the introduction of electronic tech-nology and building control systems into most facilitytypes, new opportunities exist to develop more sophis-ticated evaluation methods and procedures.

Recommendation 8. The committee encourages theuse of existing and new integrated building monitoringsystems (such as security sensors, video monitors, andtelephones) to provide data and to record occupants’feedback in response to building conditions. Researchshould be undertaken to develop on-line sensors, wearand tear indicators, and other potentially beneficialapplications of monitoring technology to be used inPOE programs.

Discussion 8. Many existing building monitoring sys-tems and technologies routinely report on ambient con-ditions or on ongoing activities in buildings. Securitycameras and movement sensors that regulate lighting,telephones, and other systems can be used to providePOE data and feedback to facility managers.

Computer-Based Systems

Trend 9. New computer-based technologies offerdesigners and end users the capability of visualizingand testing design concepts before they are actuallybuilt. They also offer more dynamic ways of sharinginformation and examining “what if” options in design.

Conclusion 9. How results of POE work or howgraphical data are presented to an untrained audience isimportant. Long, written reports or complex graphicscan lead to incorrect conclusions. Computer-basedtechnologies, including computer-aided design (CAD)systems, offer many opportunities to educate anuntrained audience and to communicate effectivelynew information.

Recommendation 9. The committee recommends thedevelopment of computer-based reporting formats thatgive POE practitioners the ability to communicateeffectively their findings to a nontechnical audience.Alternative reporting and presentation formats shouldalso be investigated.

Discussion 9. A CAD system could be linked to aPOE data base to provide feedback on particular designconfigurations and strategies. CAD-generated materialscould be reviewed and tested with prospective occu-pants before building construction begins. Psychologi-cal imaging, problem solving, and idea generationexercises could be used to augment the CAD capabilityin order to produce realistic images of actual occupancyexperiences. For example, in Japan some developersalready use CAD systems to help people design newhomes. After prospective buyers play “what if” gameswith the sales person, a particular design is selected,and a CAD system produces information for the pre-fabrication of the housing units, including schedulingand delivery to the building site.

Simulations

Trend 10. Increasingly, electronic simulations areused to provide realistic experiences of actual live situ-ations. The pilot trainer simulator is an example of thistrend.

Simulators provide airplane pilots with the actualexperiences, perceptual information, and other realis-tic inputs to simulate a situation or set of conditionsthat they might experience. Computers simulate instru-

APPENDIX F 125

mentation readings that appear in response to the pilot’scontrol changes, and video displays provide them withrealistic views of what they will encounter at differentairports.

Conclusion 10. Computer simulations, used now inother industries, will provide POE practitioners withthe tools and techniques to anticipate the findings of aPOE before the building is built and occupied, enablingthe end user to have input at the conceptual and designphases of the building process.

Recommendation 10. Computer simulations, such asfull scale and smaller scale mock-ups, should be devel-oped and used to complement POEs.

Discussion 10. Today, various psychological or othertests are used to evaluate situations such as job func-tions, living in a space capsule, or college performance.Some manufacturers have developed software pro-grams that allow the evaluation of a machine partbefore it is produced. Simulation or evaluation testscould be developed to anticipate the significant find-ings that might be uncovered by POEs. While they willnot remove the need for POEs, these simulations couldprovide critical feedback to designers.

Mathematical Modeling

Trend 11. Mathematical models are used to evaluatevarious economic, physical and political scenarios that

could develop under certain conditions or assumptions.Scientists studying weather, geological events such asearthquakes, regional ecologies, and other naturalsystems rely on mathematical computer modeling tosimulate possible outcomes and their likelihood ofoccurrence.

Conclusion 11. With availability of powerful com-puters and sophisticated software, it is becoming easierto use mathematical models for environmental designevaluations such as POEs.

Recommendation 11. Ways should be examined thatallow the use of POE in early phases of building pro-gramming and design, before a building is actuallycompleted and occupied. Simulations to replicatePOEs, mathematical or statistical formulations, orexpert computer systems may allow these types of pre-occupancy evaluations to be carried out.

Discussion 11. Models using POE information couldbe developed with the goal that at some time in thefuture the building industry will be able to do POE-type testing early in the programming phase of thebuilding process. Planners would be able to ask “whatif” questions and test them under various occupancyscenarios. Once POE findings are systematized, ascalled for in this report, it will become possible to applymathematical and statistical models in the hope thatsome better predictability in design can be achieved.

126

Bibliography

Allen, T. (1977). Managing the Flow of Technology. Cambridge, Mass.:MIT Press.

American Society for Testing and Materials. (1987). ASTM Standards onComputerized Systems. West Conshohocken, Pa.

American Society for Testing and Materials. (1999). ASTM Standards onBuilding Economics. West Conshohocken, Pa.

American Society for Testing and Materials. (2000). ASTM Standards onWhole Building Functionality and Serviceability. West Conshohocken,Pa.

Amiel, M.S., and Vischer, J.C. (1997). Space Design and Management forPlace Making—Proceedings of the 28th Annual Conference of theEnvironmental Design Resarch Association. Edmond, Okla.

Ang, G., et al. (2001). A Systematic Approach to Define Client Expectationto Total Building Performance During the Pre-Design Stage. Proceed-ings of the CIB 2001 Triennial Congress. Rotterdam, Holland.

Argyris, C. (1992). On Organizational Learning. Cambridge, Mass.:Blackwell, Business.

Argyris, C. (1992). Teaching smart people how to learn. In Argyris, C.,(Ed.) On Organizational Learning. Cambridge, Mass.: Blackwell Busi-ness. pp. 84-100.

Argyris, C., and Schon, D. (1978). Organizational Learning. Reading,Mass.: Addison-Wesley.

Aronoff, S., and Kaplan, A. (1995). Total Workplace Performance:Rethinking the Office Environment. Ottawa: WDL Publications.

Babbie, E. (1990). Survey Research Methods. Belmont, Calif.: Wadsworth.Baird, G., Gray, J., Isaacs, N., Kernohan, D., and McIndoe, G. (1996). Build-

ing Evaluation Techniques, Wellington, New Zealand: McGraw-Hill.Bainbridge, W. (1999). Cyberspace: Sociology’s natural domain. Contem-

porary Sociology 28(6):664-667.Bakens, W. (2001). Thematic Network PeBBu—Performance Based Build-

ing—Revised Workplan, Rotterdam: CIB.Balaram, S. (2001). Universal design and the majority world. In: Preiser,

W.F.E., and Ostroff, E. (Eds.) Universal Design Handbook. New York:McGraw-Hill.

Basi, R. (1999). WWW response rates to socio-demographic items. Journalof the Market Research Society 41 (4):397-401.

Becker, F., and Steele, F. (1995). Workplace by Design: Mapping the HighPerformance Workscape. San Francisco, Calif.: Jossey-Bass.

Bordass, W., and Leaman, A. (1997). Future buildings and their services:Strategic considerations for designers and clients. Building Researchand Information 25(4):190-195.

Bourdeau, Luc (Ed.) (1999). Agenda 21 on Sustainable Construction, CIBReport Publication 237. Rotterdam, Holland.

Bradley, N. (1999). Sampling for Internet surveys: An examination of re-

spondent selection for Internet research. Journal of the Market ResearchSociety 41(4):387-395.

Brager, G., Heerwagen, J., Bauman, F., Huizenga, C., Powell, K., Ruland,A., and Ring, E. (2000). Team Spaces and Collaboration: Links to theEnvironment. Berkeley: University of California, Center for the BuiltEnvironment.

Brand, S. (1994). How Buildings Learn, What Happens After They’re Built.New York: Penguin Books.

Brill, M., and Weidemann, S. (1999). Workshop presented at theAlt.Office99 Conference. San Francisco, Calif.

Brill, M., Margulis, S.M., and Konar, E. (1985). Using Office Design toIncrease Productivity (2 vols.). Buffalo, N.Y.: BOSTI andWestinghouse Furniture Systems.

Burns, P. (Ed.). SAM—Strategic Asset Management Newsletter, Salisbury,South Australia: AMQ International.

Business Week (1996). The new workplace. April 29, pp.107-117.Cameron, I., and Duckworth, S. (1995). Decision Support. Industrial Devel-

opment Research Foundation.Campbell, D.T. (1999). Social Experimentation. Sage Publications.Center for Environmental Design Research (Center for the Built Environ-

ment). (1996). Vital Signs. Berkeley: University of California.www.cbe.berkeley.edu.

Center for Universal Design. (1997). The Principles of Universal Design(Version 2.0). Raleigh, N.C.: North Carolina State University.

Centre scientifique et technique du bâtiment (1990). Améliorerl’architecture et la vie quotidienne dans les bâtiments publics Paris:Plan construction et architecture. Ministère des équipements, dulogement, des transports et de l’espace.

CIB (1996). A Model Post-Construction Liability and Insurance System,CIB Publication 192. Prepared under the supervision of CIB W087.

Cohen, R., Bordass, W., and Leaman, A. (1996). Probe: A Method of Inves-tigation. Harrogate, U.K.: CIBSE-ASHRAE Joint National Conference.

Cooper, C. (1973). Comparison Between Architects’ Intentions and Resi-dents’ Reactions, Saint Francis Place San Francisco. Berkeley, Calif.:Center for Environmental Structure.

Consumer Reports. (2000). Rating the Raters, August 31.Corry, S. (2001). Post-occupancy evaluation and universal design. In:

Preiser, W.F.E., and Ostroff, E. (Eds.) Universal Design Handbook.New York: McGraw-Hill.

Cotts, D.G., and Lee, M. (1992). The Facility Management Handbook. NewYork: AMACOM, a division of American Management Association.

Davis, G., and Altman, I. (1996) Territories at the Work-Place: Theory intoDesign Guidelines, in Man-Environment Systems, Volume 6-1, 46-53.Also published, with minor changes, in Korosec-Serfati, P. (Ed.)

BIBLIOGRAPHY 127

Appropriation of Space, Proceedings of the Third International Archi-tectural Psychology Conference at Louis Pasteur University, Strasbourg,France, 1977.

Davis, G. et al. (1993). Serviceability Tools Manuals, Volumes 1 and 2.Ottawa: International Centre for Facilities.

Davis, G. et al. (2001) Serviceability Tools, Volume 3—Portfolio and AssetManagement: Scales for Setting Requirements and for Rating the Con-dition and Forecast of Service Life of a Facility—Repair and Alteration(R&A) Projects. Ottawa: International Centre for Facilities.

Davis, G., and Szigetti, F. (1996). Serviceability tools and methods (STM):Matching occupant requirements and facilities. In Baird, G., Gray, J.,Isaacs, N., Kernohan, D., McIndoe, G. (Eds.) Building Evaluation Tech-niques. New York: McGraw-Hill.

Dilani, A. (ed.) (2000). Proceedings of the 3rd International Conference onHealth and Design. Stockholm: University of Stockholm.

Dillon, R., and Vischer, J. (1988). The Building-in-Use Assessment Meth-odology (2 volumes). Ottawa: Public Works Canada.

Eley, J., and Marmot, A.F. (1995) Understanding Offices, What Every Man-ager Needs to Know About Office Buildings. Middlesex, U.K.: PenguinBooks.

Ellis, W.R., and Cuff, D. (1989). Architects’ People. New York: OxfordUniversity.

Farbstein, J., and Kantrowitz, M. (1989) Post-occupancy evaluation andorganizational development: The experience of the United States PostalService. In Preiser, W. (Ed.) Building Evaluation. New York: PlenumPress.

Federal Facilities Council. (1998). Government/Industry Forum on CapitalFacilities and Core Competencies. Washington, D.C.: National Acad-emy Press.

Fisk, W., and Rosenfeld, A.H. (1997). Estimates of improved productivityand health from better indoor environments. Indoor Air 7:158-172.

Flagg, G. (1999). Study finds major flaws in San Francisco main library.American Libraries 30(9):16.

Friedmann, A., Zimring, C., and Zube, E. (1978). Environmental DesignEvaluation. New York: Plenum Press.

General Services Administration, Office of Governmentwide Policy, Officeof Real Property. (1997). Office Space Use Review. Washington, D.C.

General Services Administration, Office of Governmentwide Policy, Officeof Real Property. (1998). Government-wide Real Property PerformanceResults—Government-wide baseline—Private Sector Performance—Building Profiles. Washington, D.C.

General Services Administration, Office of Governmentwide Policy, Officeof Real Property. (1998). Government-wide Real Property PerformanceMeasurement Study. Washington, D.C.

General Services Administration. (1999). The Integrated Workplace: AComprehensive Approach to Developing Workspace. Washington, D.C.:Office of Governmentwide Policy and Office of Real Property.

Gibson, E.J. (1982). Working with the Performance Approach in Building.CIB Report, Publication 64. Rotterdam, Holland.

Glover, M. (Ed.) (1976). Alternative Processes; Building Procurement,Design and Construction. Occasional Paper Number 2. Montreal,Quebec: Industrialization Forum.

Grantham, C. (2000). The Future of Work: The Promise of the New DigitalWork Society. New York: McGraw-Hill, Commerce Net Press.

Gray, J. (in press) Innovative, Affordable, and Sustainable Housing. Pro-ceedings of the CIB 2001 Triennial Congress. Rotterdam, Holland.

Gregerson, J. (1997). Fee not-so-simple. Building Design and Construc-tion. August 30-32.

Guimaraes, M.P. (2001). Universal design evaluation in Brazil: Developingrating scales. In: Preiser, W.F.E., and Ostroff, E. (Eds.) UniversalDesign Handbook. New York: McGraw-Hill.

Hargreaves, D. (2001). Junk e-mail costs online surfers $14 billion a year:EU report. Financial Post. February 3, p. D9.

Hattis, D.B., and Ware, T.E. (1971). The PBS Performance Specificationfor Office Buildings. Prepared for the Office of Construction Manage-

ment, Public Buildings Service, General Services Administration, bythe Building Research Division, Institute for Applied Technology, Na-tional Bureau of Standards, U.S. Department of Commerce. Washing-ton, D.C.: NBS Report 10 527.

Heerwagen, J. (2000). Green buildings, organizational success and occu-pant productivity. Building Research and Information. 28 (5/6):353-367.

Horgen, T.H., Joroff, M.L., Porter, W.L., and Schon, D.A. 1999. Excellenceby Design: Transforming Workplace and Work Practice. New York:Wiley.

Huber, G.P. (1991). Organizational learning: The contributing processesand the literature. Organization Science 2:88-115.

International Organization for Standardization (1994). ISO 9000 Compen-dium—International Standards for Quality Management, 4th Edition.Geneva, Switzerland.

International Organization for Standardization 9000 (in process of re-edition)—Guidelines 9001 and 9004.

Joiner, D., and Ellis, P. (1989). Making POE work in an organization. In:Preiser, W. (Ed.) Building Evaluation. New York: Plenum Press.

Jones, L. (2001). Infusing universal design into the interior design curricu-lum. In: Preiser, W.F.E., and Ostroff, E. (Eds.) Universal Design Hand-book. New York: McGraw-Hill.

Joroff, M., Louargand, M., Lambert, S., and Becker, F. (1993). StrategicManagement of the Fifth Resource: Corporate Real Estate. IndustrialDevelopment Research Foundation.

Kantrowitz, M., and Farbstein, J. (1996). POE delivers for the post office.In: Baird, G., Gray, J., Isaacs, N., Kernohan, D., McIndoe, G. (Eds.)Building Evaluation Techniques. New York: McGraw-Hill.

Kaplan, R.S., and Norton, D.P. (1996). The Balanced Scorecard. Boston:Harvard Business School Press.

Kaye, B., and Johnson, T. (1999). Research methodology: Taming the cyberfrontier. Social Science Computer Review 17(3):323-337.

Kernohan, D., Gray, J., and Daish, J. (1992). User Participation in BuildingDesign and Management : A Generic Approach to Building Evaluation.Oxford: Butterworth Architecture.

King, N. (2000). What are they thinking? PC Magazine (February 8): 163-178.

Lorch, R. (Ed.) (2001). Post-occupancy evaluation (Special Issue) BuildingResearch & Information 29(2).

Knocke, J. (1996). A Model Post-Construction Liability and Insurance Sys-tem. CIB Publication 192, prepared under the supervision of CIB W087.Rotterdam, Holland.

Leaman, A., Cohen, R. and Jackman, P. (1995). Ventilation of office build-ings: Deciding the most appropriate system. Heating and Air Condi-tioning (7/8):16-18, 20, 22-24, 26-28.

Loftness, V., et al. (1996). Re-valuing Buildings: Invest Inside Buildings toSupport Organizational and Technological Change Through Appropri-ate Spatial, Environmental and Technical Infrastructures. Steelcase,Inc.

Lundin, B.L. (1996) Point of departure—Standard offers a new way todetermine if a building measures up. Building Operating Management(September):154.

Lynn, M., and Davis, G. (1998). The Need to Adopt an Auditable Commer-cial Real Estate Building Measurement Standard for Purposes of Secu-rities Disclosure (A Ticking Time Bomb). International Center forFacilities.

Markus, L. (1990). Toward a “critical mass” theory of interactive media. InFulk, J., and C. Steinfield (Eds.) Organizations and CommunicationTechnology. Newbury Park, Calif.: Sage. pp. 194-218.

Maslow, H. (1948). A theory of motivation. Psychological Review 50:370-398.

McGregor, W., and Then, D.S. (1996). Facilities Management and the Busi-ness of Space. Arnold, a member of the Hodder Headline Group.

McLaughlin, H. (1997). Post-occupancy evaluations: “They show us whatworks and what doesn’t.” Architectural Record 14.


Mill, P., and Kaplan, A. (1982). A Generic Methodology for ThermographicDiagnosis of Building Enclosures. Ottawa: Public Works Canada (nowPWGS: Public Works and Government Services Canada).

Mohr, A. (2000). The new metrics—going beyond costs per square foot tomeasure building performance. Today’s Facility Manager (February).

Morgan Stanley Dean Witter. (2001). Infoworld March 12, p.16.Nasar, J.L. (Ed.) (1988). Environmental Aesthetics: Theory, Methods and

Applications. Cambridge, Mass.: MIT Press.National Research Council. (1987). Post-Occupancy Evaluation Practices

in the Building Process: Opportunities for Improvement. Washington,D.C.: National Academy Press.

Neufeld, E. (1997). Where are the audiences going? MediaWeek May 5, pp.S22-S29.

Norman, D. (1993). Things That Make Us Smart: Defending HumanAttributes in the Age of the Machine. Reading, Mass.: Addison-Wesley.

Nua Internet Surveys. (2000). http://www.nua.ie/surveys/how_many_online.O’Mara, M. (1999). Strategy and Place: Managing the Corporate Real

Estate in a Virtual World. The Free Press.O’Mara, M. (Ed.) (2000). Special issue on portfolio management. Journal

of Corporate Real Estate 2(2).Ostroff, E. (1997). Mining our natural resources: The user as expert. Inno-

vation 16(1).Ouye, J.O. (1998). Measuring workplace performance: Or, yes, Virginia, you

can measure workplace performance. Paper presented at the AIA Con-ference on Highly Effective Facilities, Cincinnati, Ohio: March 12-14.

Ouye, J.O., and Bellas, J. (1999). The Competitive Workplace. Tokyo:Kokuyo (fully translated in English and Japanese).

Pedersen, A. (2001). Designing cultural futures at the University of West-ern Australia. In: Preiser, W.F.E., and Ostroff, E. (Eds) UniversalDesign Handbook. New York: McGraw-Hill.

Perseus. (2000). Survey 101—A complete guide to a successful survey.www.perseus_101b.htm.

Pike, P. (2001). Technology rant: I’m tired of feeling incompetent! PikeNetFebruary 11. www.pikenet.com.

Preiser, W.F.E. (1983). The habitability framework: A conceptual approachtoward linking human behavior and physical environment. DesignStudies 4(2).

Preiser, W.F.E., Rabinowitz, H.Z., and White, E.T. (1988). Post-OccupancyEvaluation. New York: Van Nostrand Reinhold.

Preiser, W.F.E. (1991). Design intervention and the challenge of change.In: Preiser, W.F.E., Vischer, J.C., and White, E.T. (Eds.) Design Inter-vention: Toward a More Humane Architecture. New York: VanNostrand Reinhold.

Preiser, W.F.E. (1996). POE training workshop and prototype testing at theKaiser-Permanente medical office building in Mission Viejo, Califor-nia, USA. In Baird, G., et al. (Eds.) Building Evaluation Techniques.London: McGraw-Hill.

Preiser, W.F.E., and Stroppel, D.R. (1996). Evaluation, reprogramming andre-design of redundant space for Children’s Hospital in Cincinnati. Pro-ceedings of the Euro FM/IFMA Conference, Barcelona, May 5-7.

Preiser, W.F.E. (1997). Hospital activation: Towards a process model.Facilities (12/13):306-315.

Preiser, W.F.E., and Schramm, U. (1997). Building performance evalua-tion. In: Watson, D., et al. (Eds.) Time-Saver Standards: ArchitecturalDesign Data. New York: McGraw-Hill.

Preiser, W.F.E. (1998). Health Center Post-Occupancy Evaluation: TowardCommunity-Wide Quality Standards. Proceedings of the NUTAU/USPConference, Sao Paulo, Brazil.

Preiser, W.F.E. (1999). Post-occupancy evaluation: Conceptual basis, ben-efits and uses. In: Stein, J.M., and Spreckelmeyer, K.F. (Eds.) ClassicalReadings in Architecture. New York: McGraw-Hill.

Preiser, W.F.E., and Schramm, U. (2001). Intelligent office building perfor-mance evaluation in the cross-cultural context: A methodological out-line. Intelligent Building I(1).

Raw, G. (1995). A Questionnaire for Studies of Sick Building Syndrome.London.: BRE Report. Construction Research Communications.

Raw, G. (2001). Assessing occupant reaction to indoor air quality. InSpengler, J., Samet, J., McCarthey, J. (Eds.) Indoor Air Quality Hand-book. New York: McGraw-Hill.

Ripley Architects. (2000). San Francisco Public Library Post OccupancyEvaluation Final Report. San Francisco: Ripley Architects.

Rondeau, E.P., Brown, R.K., and Lapides, P.D. (1995) Facility Manage-ment. New York: John Wiley & Sons.

Schaefer, D., and Dillman, D. (1998). Development of a standard e-mailmethodology. Public Opinion Quarterly 62:378-397.

Schein, E.H. (1995). Learning Consortia: How to Create Parallel LearningSystems for Organization Sets (working paper). Cambridge, Mass.:Society for Organizational Learning.

Schiller, G. et al. (1988). Thermal Environments and Comfort in OfficeBuildings. CEDR-02-89. Berkeley: University of California, Center forEnvironmental Design Research.

Schiller, G., et al. (1989). Thermal comfort in office buildings. ASHRAEJournal. 26-32

Schneekloth, L.H., and Shibley, R.G. (1995). Placemaking: The Art andPractice of Building Communities. New York: Wiley.

Sheehan, K., and McMillan, S. (1999). Response variation in e-mail sur-veys: An exploration. Journal of Advertising Research 39(4):45-54.

Shibley, R. (1982). Building evaluations services. Progressive Architecture63(12): 64-67.

Shibley, R. (1985). Building evaluation in the main stream. Environmentand Behaviour (1):7-24.

Sims, W.R., Joroff, M., and Becker, F. (1998). Teamspace Strategies: Cre-ating and Managing Environments to Support High Performance Team-work. Atlanta: IDRC Foundation.

Smith, P., and Kearny, L. (1994). Creating Workplaces Where People CanThink. San Francisco: Jossey-Bass.

Spillinger, R.S., in conjunction with the FFC (Federal Facilities Council).2000. Adding Value to the Facility Acquisition Process: Best Practicesfor Reviewing Facility Designs. Washington, D.C.: National AcademyPress.

Steele, F. (1973). Physical Settings and Organizational Development. Read-ing, Mass.: Addison-Wesley Publishing Company.

Steele, F. (1986). Making and Managing High-Quality Workplaces. NewYork:Teacher College Press, Columbia University.

Stewart, T.A. (1999). Intellectual Capital. New York: DoubledaySullivan, E. (1994). Point of Departure—ISO 9000: Global benchmark for

manufacturers may be future facility challenge. Building OperatingManagement (October):96.

Swartz, P. (1991). The Art of the Long View. New York: Doubleday.Swoboda, W., Muhlberger N., Weitkunat R., and Schneeweib, S. (1997).

Internet surveys by direct mailing. Social Science Computer Review15(3):242-255.

Taylor, H. (2000). Does Internet research work? International Journal ofMarket Research 42(1) 51-63.

Teicholz, E. (Ed.) (2001). Facilities Management Handbook. New York:MacGraw-Hill.

Thompson, H. (2000). The Customer-Centered Enterprise—How IBM andOther World-class Companies Achieve Extraordinary Results by Put-ting Customers First. New York:McGraw-Hill.

USA Today (2001). Searching the Web in native language. February 27, 7B.U.S. Department of Commerce. (1999). Falling through the net: Defining

the digital divide. National Telecommunications and Information Ad-ministration. http://www.ntia.doc.gov/ntiahome/fttn99.

Ventre, F. (1988). Sampling building performance. Paper presented at Fa-cilities 2000 Symposium. Grand Rapids, Mich.

Vischer, J. (1985). The adaptation and control model of user needs in hous-ing. Journal of Environmental Psychology 5:287-298.

Vischer, J. (1989). Environmental Quality in Offices. New York: Van

BIBLIOGRAPHY 129

Nostrand Reinhold.Vischer, J. (1993). Using occupancy feedback to monitor indoor air quality.

ASHRAE Transactions 99(pt. 2).Vischer, J. (1996). Workspace Strategies: Environment as a Tool for Work.

New York: Chapman and Hall.Vitruvius. (1960). The Ten Books on Architecture (translated by M.H.

Morgan). New York: Dover Publications.von Foerster, H. (1985). Epistemology and Cybernetics: Review and

Preview. Milan, Italy: Casa della Cultura.Watson, C. (1996). Evolving design for changing values and ways of life.

Paper presented at the IAPS14, Stockholm.Watson, D., Crosbie, M.J., and Callender, J.H. (Eds.) (1997). Time-Saver

Standards: Architectural Design Data. New York: McGraw-Hill (7th

Edition).

Welch., P. (Ed.) (1995). Strategies for Teaching Universal Design. Boston,Mass.: Adaptive Environments Center.

Welch, P., and Jones, S. (2001). Teaching universal design in the U.S. In:Preiser, W.F.E., and Ostroff, E. (Eds.) Universal Design Handbook.New York: McGraw-Hill.

Wineman, J. D. (Ed.) (1986). Behavioral Issues in Office Design. New York:Van Nostran Reinhold.

Zeisel, J. (1975). Inquiry by Design. New York: Brooks-Cole.Zeisel, J. (1975). Sociology and Architectural Design. New York: Russell

Sage Foundation.Zeisel, J. (in press). Inquiry by Design, 2nd edition. New York: Cambridge

University Press.Zimring, C.M., and Reizenstein, J.E. (1981). A primer on post-occupancy

evaluation. Architecture (AIA Journal) 70(13):52-59.

Documents

LEARNING FROM OUR BUILDINGS - Egypt Arch from... · 2011-03-15 · LEARNING FROM OUR BUILDINGS A State-of-the-Practice Summary of Post-Occupancy Evaluation Federal Facilities Council