Statistics Canada’s proposedHousehold Survey Strategy

Sylvie Michaud, Statistics Canada

Mannheim, Germany

November 14, 2007

Why a household survey strategy Too costly and too slow

Labour Force Survey – our traditional platform for ad hoc survey work -- is “maxed out”, restricting our ability to take on new surveys at reasonable cost

Increasing use of cell phones casts doubt on the future of Random Digit Dialing (RDD) surveys…

…as do declining response rates

Need to increase capacity to conduct surveys is a cost-effective way

Context : Budget review Likely every four year for every department

STC in the first phase review

Actual target is not final yet

Direct impact on us; may be on cost recovery ?

Collection infrastructure Three infrastructures for social surveys; most

interviews done with computer assisted interviewing Decentralised field collection (CAPI) Centralised under 3 Regional Offices (CATI) Ottawa (small)

Collection done using Blaise Different management systems for different

environments

Research to add efficiency on collection process : outside HSS

SLID 2006 - Distribution of Attempts and Time Spent by Data Collection Phase

18%

35%

8% 7%

22%

10%10%

29%

37%

4%

13%8%

0%

10%

20%

30%

40%

50%

Before f irstcontact

Betw een firstcontact &interview

Interview Before f irstcontact

After the f irstcontact

Nevercontacted

Response Non-Response

% Attempts

% Time Spent

Stock-taking: overview of existing survey program Study in 2005 on size, scope, cost of household survey

program

Based on about 15 surveys

All used samples drawn from households

Either base funded or recurring cost-recovery

Scope: 5 monthly surveys Labour Force Survey (LFS)

Canadian Community Health Survey (CCHS)

General Social Survey (GSS)

Travel Survey of Residents of Canada (TSRC)

Canadian Tobacco Use Monitoring Survey (CTUMS)

Scope: annual and other surveys Annual

Survey of Household Spending (SHS) Canadian Internet Use Survey (CIUS) Residential Telephone Services Survey (RTSS)

Less frequent (examples) Survey of Financial Security (SFS) Adult Education and Training Survey (AETS) Survey of Household Energy Use (SHEU) Survey of Labour and Income Dynamics (SLID)

Some facts about these surveys To collect the data, we contact about 300,000

households per year

Once selected, some households are contacted repeatedly (LFS)

About one third of budget comes from external sources

About two thirds of budget goes to data collection activities

Foundations for the strategy Efficient use of field capacity

Frames for social surveys

Governance

Flexibility for contacting respondents

Spreading interviewing workload and response burden Uneven interviewing workload increases cost – annual

cycle of hiring, training, releasing interviewers to cope with peaks

Also between-year fluctuations

Over-burdened respondents cannot realistically be recontacted – and this is key to increasing capacity

Measures already taken: Redesign of Survey of Household Spending Uniform monthly workload for Canadian Community

Health Survey

Annual distribution of field workload

010000

2000030000

4000050000

6000070000

80000

j f m a m j j a s o n d j f m a m j j a s o n d

current

future

Frames for supplements Proposal to remove TSRC from LFS, to increase capacity TSRC is constant LFS companion:

Every month 1/3 of LFS sample gets TSRC One person per household, non-proxy

Feeds SNA (inter-provincial flow tables of domestic travel expenditures), TSA, provincial monitoring of tourism trends

Since inception in 1979, survey has consumed growing share of LFS capacity: Biennial annual Quarterly monthly One sixth of LFS sample one third

Issues with frames

Costly to run supplements because no capacity for supplements to LFS

RDD may not be viable in medium term

Difficult to select specific sub-populations

Frames updates are costly and might not be efficient

LFS supplementary survey program before and after TSRC

# topics # labour related topics

1978 9 4

1979 8 5

1984 10 7

1989 11 5

1993 11 5

2005 3 0

2006 3 0

TSRC removed from LFS supplement should bring capacity TSRC as a LFS supplement hinders capacity to quickly

and efficiently conduct supplements related to Labour Market and synergy in topic not there

Pilot project May 2008 Fresh sample for TSRC

Varying recall periods Longitudinal design If this was done, this would re-establish capacity to conduct

supplements

Timing of the redesign is an issue

Integrated household survey platform

Core content

CCHS

Core content

SHS & Supplements

Core content

LFS & Supplements

Core content

TSRC

Master Sample(Approx 260K hhlds per year, with standard core content)

Core content

GSS

Ad hoc /smallersurvey

Ad hoc /smaller survey

Ad hoc/smallersurvey

Ad hoc / smallersurvey

Survey integration Identical core questions for all surveys (demographics,

etc)

Create a master sample using households exiting from “1st phase” surveys

Draw on master sample for other (2nd phase) surveys

Harmonized content modules (18 groups of variables) for new surveys

Master sample integration

Pilot project April – May 2008 Conducted on an on-going survey currently

done with RDD Targetting population people aged 45 and over

Test logistics but also impact on estimates from the two methodologies

Sampling frames Address Register becomes frame for selecting

dwellings in urban areas (~65% of sample) Dual frame in rural areas

Telephone list frame Area frame

Dual frame combines quality of area frame with cost-effectiveness of telephone list frame

All innovations already partially in use: LFS has used Address Register CCHS has used telephone list frame in a dual frame

context

AR

GEO(NGD)

LFS,etc.

Updates(admin, listing)

permanent

permanent

NEW

Links frames

Governance - internally Standards

Content system

Interactions with collection Links with Census

Organisationnal impact ?

Governance – Testing tripartite governance for longitudinal

surveys

Statistics Canada, policy departments, researchers

Across surveys ?

Northern strategy Opportunity to examine options for best

approach in North

Aim is to manage burden and yield data that are as useful as possible

Co-development approach, involving territorial governments, other stakeholders and Statcan

Contact strategy Need to be multi-mode

Mode of choice varies depending on the age of respondents

Trends are changing

Cell phone only is increasing, along with Voice over IP

Which way would like to be contactedenvironics survey n=1965

0

10

20

30

40

50

60

70

80

90

mail telephone internet visit

20052007

Survey methods most preferred: respondents who prefer more than one wayenvironics

No preference

In-person

Telephone

Paper

Internet26

54

39

25

26

15

7

5

1

1

2005

2007

Future Census completion – mode preference By ageenvironics n=1965

0% 20% 40% 60% 80% 100%

18-29

30-44

45-59

60+

internetpaperdk

Internet option for respondents Build on success of Census Internet option Assumptions:

1st interview is interviewer-administered Respondent can opt for Internet after that Fallback to CATI Short supplements (with proxy) still viable

Two pilots with internet in November Need to build a more robust master control system

Use of other information Administrative files:

How far and what is respondents perception ? How does it impact on access ?

Use of Census Geographic information As a frame As a data source for longitudinal analysis ?

Summary

We started two years ago We wanted to position our strategic

thinking for the next five years Likely not be finished in year 5 however

Starting to see some cost savings with load levelling but it’s more than costs savings

Survey of Household Spending

Monthly collection Key variables collected from all households (30

minutes) Sample then divided into two subsamples, to respond to

a subset of more detailed spending questions Recall periods defined according to the type of

expenditure Capacity for rotating supplementary content (wealth,

health, environment…)

Lining up activities and directions with original objectives Increased capacity

Master sample More monthly surveys LFS capacity freed up

Improved cost-effectiveness Increased use of AR & new sampling strategies Interviewer workload more uniform Harmonized content modules

Faster turnaround times Harmonized content modules

Lining up activities and directions with original objectives Increasing prevalence of cell phones

Dwelling-based approach

Rising non-response rates Multi-modal approach: more options for respondents Burden spread more uniformly (and managed)

declining response rates : Survey of Labour and Income Dynamics

Wave

Panel

1 2 3 4 5 6

1993 93.3 89.6 86.5 83.9 82.4

81.5

1996 89.5 86.7 85.2 82.7 78.5

77.4

1999 83.9 83.0 83.0 79.6 76.4

73.7

2002 81.2 83.2 78.3 75.0 … …

2005 78.8 … … … … …

Context :Households with cell phone only

Nationally, 90% have a land line

Households with cell phones only increased from 1.9% to 5% in past 3 years

10.6% use voice over IP

Rate for low income household almost twice that of other households

Household survey strategy Spread interviewer and response burden

Survey integration

Increase response options

Greater use of list frames for sampling