1

Framework for Prioritizing Framework for Prioritizing Economic Statistics ProgramsEconomic Statistics Programs

This report is released to inform interested parties of research and to encourage discussion. Any views expressed on statistical, methodological, technical, or operational issues are those of the authors and not necessarily those of the U.S. Census Bureau.

Presented by Thomas L. Mesenbourg Associate Director for Economic ProgramsThomas.L.Mesenbourg.Jr.@census.gov

June 2007

2

Presentation OutlinePresentation Outline

• Budget Environment

• Framework Purposes

• Economic Program Criteria

• Uses of Criteria

• Program Improvements Model

• Initial Findings

• Conclusions

3

Budget EnvironmentBudget Environment

• Tight resources for remainder of decade• Budget cuts will lead to program

eliminations• Even if resources are constrained

program, improvements must be made• Most program improvements will have

to be funded internally

4

Framework PurposeFramework Purpose

• Help prioritize programs• Provide information for responding to

budget cuts• Facilitate reallocation decisions

– identify costs savings– identify relative

efficiencies/inefficiencies• Ensure program decisions are more

data-driven

5

Economic Program Criteria Pre - 2006Economic Program Criteria Pre - 2006

• Retain programs providing source data to BEA and FRB

• Preserve programs and content that serve as benchmarks for GDP and other measures of economic activity

• Preserve data quality of existing programs

6

2007 Program Priorities2007 Program Priorities

• 2010 Decennial Census• Economic Census and Census of

Governments• Principal Economic Indicators and

related annual surveys• Surveys that provide source data for

NIPA• Remaining surveys not directly used

by BEA

7

Initial Use of New Criteria in FY 2007Initial Use of New Criteria in FY 2007

• Needed to identify $10 million in program cuts• Priority 2 – Economic Census and Census of Governments

– Survey of Business Owners – suggested eliminating coverage of businesses with no paid employees

• Priority 3 – Principal Economic Indicators and related annual surveys– Quarterly Financial Report program – suggested eliminating

coverage of small manufacturers• Priority 4 – Economic surveys providing NIPA source data Suggested eliminating:

– Information and Communication Technology Survey– Current Industrial Reports– Survey of Residential Alterations and Repairs

• Priority 5 – All other– County Business Patterns – suggested one year suspension

8

Program Improvement Framework Program Improvement Framework Model DevelopmentModel Development

• Framework was developed over a year ago• Ranks programs using various attributes

such as relevance, cost effectiveness, users and uses, and quality

• Methodology is still in its infancy and needs additional refinements

• Nine annual programs and eight economic indicators included in the model

• Programs account for some $82 million or about 60% of our current programs budget

9

Metrics UsedMetrics Used

• Survey value – Our two most important stakeholders, BEA and the Federal Reserve Board were asked to rank the surveys in terms of importance

• Data quality – The survey’s unit response and coefficient of variation or CV for its principal variable were used for data quality

• Cost efficiency – Two measures for cost efficiency were used, cost per annualized number of survey units and cost per annualized number of collected variables

10

Metrics Not UsedMetrics Not Used

• Extent of GDP coverage

• Customer satisfaction from our annual web survey

• Number of web page hits

• Timeliness of publication

• Average revision size

11

Indicator FindingsIndicator FindingsProgram BEA FRB Response CV $/Unit $/Variable

New Residential Construction 5 4 5 3 5 5

Value of Construction Put in Place 5 4 2 4 5 4

Advanced Monthly Retail 5 4 1 5 5 2

Monthly Retail Trade Survey 5 4 2 5 4 3

Monthly Wholesale Survey 5 4 3 4 4 4

Manufacturers’ Shipments, Inventories, and Orders Survey

5 4 4 NA 3 3

Quarterly Financial Report 4 5 2 5 2 5

Quarterly Services Survey 5 4 1 3 2 1

Average 4.9 4.1 2.5 4.1 3.8 3.4

12

Annual FindingsAnnual FindingsProgram BEA FRB Response CV $/Unit $/Variable

Information and Communication Technology Survey

4 4 3 5 5 5

Annual Capital Expenditures Survey 5 4 3 4 5 5

Current Industrial Reports 5 5 3 3 4 3

Annual Public Employment Survey 5 3 4 5 4 5

Service Annual Survey 5 4 4 3 4 3

Annual Survey of Manufacturers 5 5 5 5 4 4

Annual Retail Trade Survey 5 4 4 5 3 3

Annual Government Finance Survey 5 3 4 5 3 4

Annual Trade Survey 5 3 4 4 1 1

Average 4.9 3.9 3.8 4.3 3.7 3.7

13

Initial FindingsInitial Findings• Assessment of individual programs can not

be captured by a single measure• Programs must be evaluated using multiple

dimensions including some not considered• BEA and FRB rankings support our criteria,

but were not very useful in identifying lower priority programs

• Response rate measures strikingly different between indicators and annual surveys

• Efficiency measures not perfect, but significant differences warrant further investigation

14

ConclusionsConclusions

• Economic programs are meeting the needs of BEA and FRB quite effectively

• Any deep program cuts will significantly impact BEA source data

• No obvious programs to eliminate, collection and processing efficiencies offer some hope for funding future program improvements

• Leveraging existing surveys to collect new content is most cost effective approach