Do Economic and Demographic Characteristics Differ between Web and Mail Respondents to the 2005 Census of

Agriculture Content Test?

By Nancy J. Dickey

Zulma T. Riberas

National Agricultural Statistical Service (NASS) Census of Agriculture

Census is conducted every 5 years Prior to the census a content test is

conducted

Why conduct a Census Content Test?

To evaluate:• Format and design of the instrument• New content items• Changes to question wording• Procedural changes

Why conduct a Census Content Test?

Results are used to identify modifications to incorporate into the final design of the data collection tool

What’s New for Census 2007

NASS was required to provide respondents the option of completing the census on the Web

Two paper versions of the questionnaire, a long form of 24 pages and a short form of 12 pages, and one Web version

Short form used a open table format to collect data. Respondents wrote in their crops and/or livestock information

SAMPLE

27,900 cases– 14,995 long form– 12,905 short form

Additional Sample

An additional sample of 200 cases received a pre-survey letter via mail encouraging them to complete the survey on the Web

These 200 were selected from a population of respondents who had previously reported an agricultural survey using a Web instrument

Survey Details

Initial mail-out was on December 30, 2005 Postcard/reminder was sent on January 17,

2006 Non-response replacement questionnaire

was sent to all non-respondents on February 9, 2006

Survey Details - Continued

Forms were checked-in, scanned for image, and data were keyed from the electronic image

Records were not edited for consistency or missing data

There was only one Web instrument which was similar in content and format to the long form paper version

Survey Results

618 respondents submitted their responses using the Web instrument

12,596 respondents submitted their responses using the mail version of the instrument

From the sample of 200, 61 responded via Web and 91 responded via mail

So, based on the 2005 Census of Agriculture Content Test did Mail Respondents differ from Web Respondents economically or demographically?

Size of Farm

Average size of the farm for Web respondents was 1,157 acres. The median size was 85 acres

Average size of the farm for mail respondents was 804 acres. The median size was 110 acres

Graph 1: Size of Farm

51.47

10.17

6.07

3.72 3.33

25.24

47.39

16.54

8.48

5.58

3.2

18.81

0

10

20

30

40

50

60

0 - 99 100 - 199 200 - 299 300 - 399 400 - 499 500 +

Acres

Percent

WEB (n = 511)

MAIL (n = 11,879)

Have Internet access?

Respondents were asked the following question:

At any time during 2005, did this operation have Internet access?

___ YES ___ NO

Have Internet access

Use of a Computer for the Farm Business ?

Respondents were asked the following question:

At any time during 2005, did this operation use a computer for the farm business?

___ YES ___ NO

Use of a Computer for the Farm Business

Age of the Principal Operator

Average age of Web respondents is 52 years Average age of Mail respondents is 58 years

Age of Operator

Age of Principal Operator

0.59

6.3

16.53

36.81

28.54

11.22

0.65

2.65

10.85

27.1527.97

30.72

0

5

10

15

20

25

30

35

40

< 25 25 - 34 35 - 44 45 - 54 55 - 64 65 +

Age Range

Perc

en

tWEB (n = 508)

MAIL (n = 11,012)

Other Economic and Demographic Comparisons

Total Household Income Type of Organization Sex of Principal Operator Spanish, Hispanic, or Latino Origin

Observations

There was no relationship between household income, type of organization, sex of principal operator, and whether the respondent was of Spanish, Hispanic, or Latino origin and the mode of data collection

Missing Data

There was a large amount of missing data for some of the items

Early in the data collection process it was discovered that some items in the web instrument were not being captured properly

These problems were fixed as they were discovered

This was the first use of the Web instrument

Missing Data – Type of Organization

52.91 percent of Web respondents did not answer this question

8.58 percent of mail respondents did not answer this question

Question was at the end of the form Respondents may have quit the Web survey

before reaching this question

Internet Access

For Web respondents, 85.76% did not answer this question

For Mail respondents, 6.80% did not respond Finding is inconsistent since Web

respondents were filling the form out on the Web

What went wrong?

Respondent was asked if the operation had access to the internet. Maybe the operator has home internet access but not specifically for the farm. So it could be the wording of the questionnaire

Summary

Analysis was based on the hypothesis that Web respondents differ from mail respondents. We expected Web respondents to be younger, male, and less racially diverse than mail respondents. We expected Web respondents to have larger operations, and greater income.

What now?

Our findings indicate that age, race, size of farm operation, internet access, and computer use for farm business seem to have a relationship to the mode of data collection chosen by the respondent

More research is needed to determine the nature of the relationship between these variables

It was disturbing to notice the high item non-response

These findings stress the importance of testing the Web instrument before respondents are given the opportunity to use it

2007 Census of Agriculture

The same analysis will be done using data from the 2007 Census of Agriculture

Knowing if certain characteristics of the population will report via the Web can result in cost savings, better data quality, and increased survey efficiency