Formulas, Ratios, Estimates, and Counts… The Historical Roots of Quantitative Public Policy in the...

Formulas, Ratios, Estimates, and Counts…

The Historical Roots of Quantitative Public Policy in the U.S

Margo AndersonHistory & Urban Studies

University of Wisconsin – Milwaukee margo@uwm.edu

2

Outline and Themes

• The federal statistical system today is wrestling with challenges to produce timely, accurate, efficient, and intelligible demographic and economic data for public policy.

• It’s an old problem.

• A brief historical survey reminds us that deploying quantitative metrics for public decision making dates from the very beginning of the republic and shapes the world we live in today.

3

Federal Statistical System Today

• Decentralized: Census, BLS, NASS, NCHS, NCES, BJS, etc.

• 98 agencies with statistical activities; 13 lead agencies

• Federalized: states and local governments also provide data through coordinating arrangements: e.g., vital statistics

• Partnerships: with research organizations and universities to collect and produce data products

4

The Statistical System within the Federal Government

• Chief Statistician resides in OMB and coordinates the system through “forms clearance” and budget authorizations.

• Legislative grounding is in the Paperwork Reduction Act of 1995.

• A diverse system.• A public system, grounded in the political

and policy making activities of the nation.

5

Federal Statistical System

• Overall budget of about $6.8 billion/year (exclusive of decennial census).

• About 40% of expenditures in 13 lead agencies• Current challenges:– Respondent cooperation.– Fiscal resource constraints.– International comparability.– More detailed data– Access to data– Statistical agency independence– Human capital – next generation…

6

Budgets: Large Agencies (millions of $)

7

Origins and Structure

• History helps! – The 1787 Constitution created the platform of the

system.– The Constitution created two different types of

statistical or public data collection and reporting: • the decennial census • the reports on government revenue and

expenditures

8

Understanding the Statistical System

• As embedded in the larger political, social, economic and demographic situation of the US

• As shaping the larger political, social, economic and demographic situation of the US

• As a technical system

9

Useful Distinctions

• Survey Data: Data collected for research or policy purposes only, usually sampled: CPS, SIPP, ACS

• Administrative Data: Data collected for administrative functions and then reused or reorganized for statistical data analysis: state unemployment records; tax records, property records, medical records.

10

Institutional and Some Technical History

• “Constituting” the system: 1780s• Implementing the system in the long 19th

century, 1790- early 1900s.• The centralization/coordination debate, 1900-

1940s• The modern system

11

Building the American State

12

13

Article 1, Section 2, of the U.S. Constitution

• "Representatives and direct Taxes shall be apportioned among the several States which may be included within this Union, according to their respective Numbers….The actual Enumeration shall be made within three Years after the first Meeting of the Congress of the United States, and within every subsequent Term of ten Years, in such Manner as they shall by Law direct."

14

Importance of the Census

• The United States was the first nation in the history of the world to take a population census and use it to allocate seats in a national assembly according to population.

15

Implementing the System

• First census was taken in 1790. • The House of Representatives and Electoral

College was first reapportioned in 1792• Immediately, government officials and the

general public recognized the significance of the new system for allocating representation.

16

Pitcher Commemorating the 1790 Census

17

Census Publications

18

Francis Edmonds, Taking the Census, 1853

19

“The Great Tribulation,” The Saturday Evening Post,

1860

20

Importance of the Census

• The U.S. has had one of the most demographically dynamic and diverse populations in the history of the world.

• The combination of the census as mechanism to adjust power and resources each decade, in conjunction with the demographic dynamism and diversity, made the census and the statistical system truly central to the functioning of the society and state

Importance of the Census

• Dynamism is measured by patterns of population growth and change

• Diversity involves geographic diversity, group diversity, and different rates of change for different parts of the country, and among the groups.

• Hence three levels– Numerical growth– Geographic diversity– Racial and ethnic diversity

21

Numerical Growth

23

From 3.9 million to 314 million

• 13 states have become 50 states.• House of Representatives grew from 65 to 435

members.• The average congressional district today is larger

than the total population of any of the original 13 states in 1790.

• Growth has been differential: some states and local areas lose while others gain.

1790

1800

1810

1820

1830

1840

1850

1860

1870

1880

1890

1900

1910

1920

1930

1940

1950

1960

1970

1980

1990

2000

0

50

100

150

200

250

300

Population Growth, 1790-2000

UK

FR

US

Year

Po

pu

lati

on

(m

illi

on

s)

25

Admitting States to the Union and Growing the House of Representatives

Differential Population Growth: New York State Population and House

Delegation, 1790-2010

26

How Is Growth Managed?

• Decisions to be made on….– Size of the House of Representatives– Admission of new states to the union– Rules for creating congressional districts

• Apportionment Formulas:– Hamilton’s Method– Jefferson’s Method– Vinton’s Method– Webster’s Method (major fractions)– Hill’s Method (equal proportions)

Geographic Diversity

Geographic Diversity: Westward Expansion

29

Geographic Diversity: The First Gerrymander, 1812

30

31

Racial and Ethnic Diversity

Article 1, Section 2, Paragraph 3

• Representatives and direct Taxes shall be apportioned among the several States which may be included within this Union, according to their respective Numbers, which shall be determined by adding to the whole Number of free Persons, including those bound to Service for a Term of Years, and excluding Indians not taxed, three fifths of all other Persons. The actual Enumeration shall be made within three Years after the first Meeting of the Congress of the United States, and within every subsequent Term of ten Years, in such Manner as they shall by Law direct.

Why were these provisions put in the Constitution?

• The framers had to decide the basis for allocating: – representation; and – revenue obligations among the states

• The rule: – revenue should be apportioned according to

ability to pay or wealth; – representation should be apportioned according

to the number of people in the state

Federal Ratio or Three Fifths Rule

Undoing the Original Constitutional Framework

• Abolishing Slavery: 13th Amendment• Amending the Constitution: Wartime

Amendments: 14th and 15th Amendments• Breaking the Link between Taxation and

Representation: 16th Amendment

Constitutional Revisions

• The Thirteenth Amendment (1865) abolished slavery and the ‘federal ratio.’ The amendment ended the discounting of the formerly slave population in the allocation formula.

• Abolition increased the level of representation of the former slave states in the House of Representatives and Electoral College and led to further constitutional revisions.

Constitutional Revisions, New Ratios

• Fourteenth Amendment, Section 2 (1868):– “Representatives shall be apportioned among the several

States according to their respective numbers, counting the whole number of persons in each State, excluding Indians not taxed. But when the right to vote at any election for the choice of electors for President and Vice-President of the United States, Representatives in Congress, the Executive and Judicial officers of a State, or the members of the Legislature thereof, is denied to any of the male inhabitants of such State, being twenty-one years of age, and citizens of the United States, or in any way abridged, except for participation in rebellion, or other crime, the basis of representation therein shall be reduced in the proportion which the number of such male citizens shall bear to the whole number of male citizens twenty-one years of age in such State.”

Constitutional Revisions

• Fifteenth Amendment (1870): “The right of citizens of the United States to vote shall not be denied or abridged by the United States or by any State on account of race, color, or previous condition of servitude.”

• Sixteenth Amendment (1913): “The Congress shall have power to lay and collect taxes on incomes, from whatever source derived, without apportionment among the several States, and without regard to any census or enumeration.”

40

First Reading of the Emancipation Proclamation

Civil War Demographic Map

42

Administrative History of the Census, 1790-1902

• From 1790 to 1902, a temporary agency in the Department of State or Interior.

• Until 1880 the US marshals and their assistants served as the field staff.

• Over the years, Congress added the collection of agricultural, manufacturing, mortality, disability statistics to the decennial.

• A very large administrative operation during the census period, but administrative discontinuity.

• Congress considered proposals for a permanent census office but did not act on them until 1902.

Who Decides?

• Congress!

44

Meanwhile….

• The other constitutionally mandated “leg” of the system developed.

45

Economic and Administrative Statistics

• Article 1, Section 9: “a regular Statement and Account of the Receipts and Expenditures of all public Money shall be published from time to time.”

• Article 2, Section 3: The President “shall from time to time give to the Congress Information of the State of the Union and recommend to their Consideration such Measures as he shall judge necessary and expedient.”

46

Implications….

• Administrative records of the revenue and expenditure of government were collected and published, making it feasible to develop administrative statistics.

• The government created an administrative structure to collect, analyze and publish the data.

Private Publications (Partnering) of Federal Statistics Begin Very Early! • Timothy Pitkin, A Statistical View of the Commerce of

the United States of America (1816)

• Adam Seybert, Statistical Annals: Embracing Views of the Population, Commerce, Navigation, Fisheries, Public Lands, Post-Office Establishment, Revenues, Mint, Military and Naval Establishments, Expenditures, Public Debt and Sinking Fund, of the United States of America, Founded on Official Documents, 1789-1818

47

48

Routine Statistical Reporting Started in the Treasury Department

• 1820: The Secretary of the Treasury began to prepare annual statistical accounts of the commerce of the US with foreign countries.

• 1840-1860s: Congress authorized hiring of clerks, regular publication of reports.

49

The Treasury Department and Permanent Statistical Offices

• 1866, Bureau of Statistics established in the Treasury Department.

• 1878, the Bureau of Statistics published the first edition of the Statistical Abstract of the United States.

50

Statistical Agencies Established in Other Departments

• Agriculture Department: 1862• Bureau of Education: 1867• Bureau of Labor: 1884• Immigration Statistics: collected in the

Treasury Department and State Department

At the Dawn of the Twentieth Century…

• The United States was recognized as a pioneer in statistical methodology and technology: machine tabulation of the census began in 1890.

• Congress had been successfully reapportioned 12 times, 46 states were in the union, and legislatures had learned to redistrict on the basis of geographic growth and change.

• Routine, reliable data poured out from federal statistical offices, guided policy development on the tariff and taxation, immigration policy, disability, labor relations, and many more areas.

The World of Printed Reports:

Statistical Abstract, 1902,

580 pages

52

53

54

New Challenges to the Statistical System

• Industrial Revolution• Rapid City Growth • Growth of the Working Classes• Disparities between Rich and Poor • Problems of Boom and Bust: Panics,

Depressions and Crashes: 1857; 1873-77; 1893-97; 1907; 1921

Examples of the Development of New Data Series…

• Price, Expenditure, and Cost of Living Measurement

• Measuring Unemployment and the Business Cycle

56

Cost of Living Measurement

57

58

Followed by Cost of Living Survey Series

• The Cost of Living Survey series was created by the United States Department of Labor, Bureau of Labor Statistics (BLS), with the goal of estimating the cost of living of a "typical" American family. – COST OF LIVING OF INDUSTRIAL WORKERS IN THE

UNITED STATES AND EUROPE, 1888-1890 (ICPSR 7711),

– COST OF LIVING IN THE UNITED STATES, 1917-1919 (ICPSR 8299)

59

Characteristics of the Tradition

• Household based survey• Relatively small number of cases• Huge number of variables• Sampling structure is complex• Conducted relatively infrequently

Measuring the Business Cycle and Unemployment

• Business cycles are irregular and frequent• They are characterized by seasonality as well

as periodicity• They call for different measurement strategies• And raise questions: is the unit of analysis:– The firm?– The household?– The overall economy?

Strategies….for Unemployment Measurement

• Administrative Data: – Labor Exchanges– Industry Reports of Activity

• Census, 1880-1910; 1930– Problems:• Infrequent data• Conceptual definitions

Mary Van Kleeck on Measuring

Unemployment, 1923

Business Cycles and Unemployment….

• “If the facts [data on employment and unemployment] are to be useful …they must be widely enough scattered geographically not to be over-influenced by condition which may be merely local in one section of the country; they must be made available by some central agency which can correlate and interpret them; and, perhaps most important of all, they must be made public with sufficient promptness to be approximately true measures of the state of employment at the time when they are issued. Thus the problem of extending and improving employment statistics is less statistical in its nature than it is administrative. It demands a machinery strong enough and simple enough to work smoothly and rapidly without breakdowns.”

Was the Statistical System up to the Task?

• The statistical system in the early 20th century was somewhat anarchic and duplicative.

• Many of the thornier issues of the Constitutional era continued to plague the statistical system, what might be called a “politics of population.”

• In 1903, with the Organic Act creating the Department Commerce and Labor, Congress mandated consolidation and coordination.

• It failed.

67

Coordination and the Politics of Population

• Theodore Roosevelt’s problem: merging agencies with traditions of administrative data and survey data.

• Herbert Hoover’s problem: the reapportionment battles of the 1920s, the only time in the history of the republic that Congress refused to reallocate House seats among the states on the basis of the census results.

• Franklin Roosevelt’s problem: the Great Depression and measuring unemployment.

69

Trying Again…and Learning to Live with Decentralization

• Bureau of Efficiency, 1920s• COGSIS, Committee on Government Statistics

and Information Services, 1930s• 1940: Position of Chief Statistician created in

the consolidation of the Bureau of the Budget (now OMB)

• 1942: Federal Reports Act mandated “forms clearance.”

70

Stuart A. Rice, first Director of Statistical Standards

71

Meanwhile…• New Deal measures such as the Social Security Act of 1935 and

the Fair Labor Standards Act (1937) created new administrative mechanisms to – Provide old age, unemployment, and disability benefits– Set standards for minimum wages and maximum hours

• The statistical sciences went through revolutions in measurement techniques and theory in sampling and inferential statistics

• The decentralized system built the National Income and Products Accounts, developed survey methods to measure employment and unemployment, and compiled an array of new administrative records data systems from the benefit programs.

Unemployment, 1890-1990

1890 1896 1902 1908 1914 1920 1926 1932 1938 1944 1950 1956 1962 1968 1974 1980 19860

5

10

15

20

25

30

35

CiviliianNon Farm

ICPSR Cost of Living Survey Series

• COST OF LIVING OF INDUSTRIAL WORKERS IN THE UNITED STATES AND EUROPE, 1888-1890 (ICPSR 7711),

• COST OF LIVING IN THE UNITED STATES, 1917-1919 (ICPSR 8299), and

• STUDY OF CONSUMER PURCHASES IN THE UNITED STATES, 1935-1936 (ICPSR 8908).

73

Followed by…Consumer Expenditure Survey Series

• The Consumer Expenditure Survey (CES) series… provides a continuous flow of information on the buying habits of American consumers and also furnishes data to support periodic revisions of the Consumer Price Index. – (1) a quarterly Interview Survey in which each consumer

unit in the sample is interviewed every three months over a 15-month period, and

– (2) a Diary Survey completed by the sample consumer units for two consecutive one-week periods.

74

The Changing World of Data Analysis

• Through 1950s, only government agencies had the capacity (technical and financial) to collect and process replicated, geographically complex and variable rich data sets.

• Academic or private research was limited to secondary analysis or significantly smaller data collections.

75

The Changing World of Data Analysis: 1960s

• The computer revolution began the process of the ‘democratization’ of data

• Federal government began production of “public use” microdata files

• Federal government proposed the creation of a “data bank” for integration, and coordination of and research using federally collected data.

• Research in “think tanks” and “contract” organizations expanded the reach of the statistical system

76

The World Gets Complicated Again…

• The War on Poverty and Great Society programs expanded upon the New Deal, 1960s-1970s– Official poverty measure created– Medicare and Medicaid programs instituted– Aid to education– Clean Air and Clean Water Acts– Revenue sharing, 1972-86

• The reapportionment revolution and civil rights movement restored focus on the original formulas and ratios for apportioning political power

77

Pressures on the Statistical System

• Small area data – for redistricting, revenue sharing allocations

• Measuring program effectiveness and compliance– SIPP; civil rights compliance

• Guaranteeing census accuracy

• And since the 1980s…anti tax movements…

21st Century Challenges…

• Measuring and making policy on global warming and climate change

• Combatting the threat of terrorism• Monitoring the economic system and

preventing another “great recession”

Lessons from the Past?

• Science operates on its own rhythm, but requires resources and legitimacy of the state to bring scientific resources/knowledge to bear on real life policy

• The political system must want a problem to be solved, which may include being willing to put the monitoring, analytic systems in place to solve the problem.

Lessons…

• For both poverty measurement and unemployment measurement, the statisticians had a pretty good idea about what to do and how to do it, but they had to wait for the political system to acknowledge the issue.

• Politicians are reluctant to embed a metric in a policy framework, particularly an automatic trigger one, like the census apportionment process.

• But when they do, expect the policy and the metrics to be revisited.

82

Thank you. For more information…

Margo AndersonHistory Department, University of

Wisconsin Milwaukeehttp://www.uwm.edu/~margo

margo@uwm.edu