Upload
zavier-tatham
View
220
Download
0
Tags:
Embed Size (px)
Citation preview
2010 Research on RespondentstVox DataMay, 2010
© Harris Interactive
Presentation to MRIA Ottawa Chapter
Doug Anderson, SVP Public AffairsHarris/Decima160 Elgin StreetSuite 160Ottawa, ON K2P [email protected]
Richard Leigh-Bennett, CMRP SVPHarris/Decima160 Elgin StreetSuite 160Ottawa, ON K2P [email protected]
What do we know – Response Rates
• The PMRS Response Rate Committee measured refusal rates in 1995, 1999, 2002 and again in 2005 and found that refusal rates have increased and response rates have fallen.
• When analyzed on an increment basis year by year, the 2002 survey suggested that for one-time studies, the rate of refusals was accelerating.
• Telephone surveys have been under attack recently on the grounds that “Results are no longer accurate nor representative” and low response rates are cited as the reason.
© Harris Interactive
Non-response Rate
• “The collective body of empirical work suggests no consistent relationship between response rate and non-response bias. RDD telephone surveys with low response rates generally still have excellent demographic representativeness.” (Keeter et al. 2006).
Questions we had….• With declining response rates, are we just getting the same
respondents who do multiple surveys or are we actually getting different people but they are harder to reach?– Number of calls received asking to do a phone survey and
number of times agreed to do the survey in past year. – Likelihood to complete a phone survey in the next year.
• Online coverage is coming close to telephone coverage, but who is doing online surveys?– Number of times participated in any sort of on-line survey.– Membership in an on-line panel.– Likelihood to complete an online survey in the next year.
Methodology
• Data collection for this study was conducted via CATI ….
• Data collection for this study was conducted April 29th to May 11th, 2010.
• Findings are based on a sample of 2008 adult Canadians. A sample of this size yields a margin of error of +/-2.2% nineteen times out of twenty.
© Harris Interactive
Findings
Estimated Requests and Interviews (12 Months)
© Harris Interactive
Reasons for Declining Telephone Survey Request
© Harris Interactive
Base: Those who declined a telephone survey (n=959)
Reasons for Accepting Telephone Survey Request
© Harris Interactive
Base: Those offering a reason (n=1,642)
Number of Calls Received
32%
28%
30%
22%
18%
32%
42%
22%
28%
32%
30%
42%
43%
35%
44%
33%
40%
44%
39%
41%
39%
40%
19%
19%
21%
20%
29%
19%
12%
27%
21%
18%
20%
7%
10%
14%
14%
21%
9%
3%
12%
10%
11%
10%
0% 20% 40% 60% 80% 100%
55+
35-54
18-34
B.C.
Prairies
Ontario
Quebec
Atlantic
Women
Men
Overall
Only This Call 2 to 4 5 to 10 11+
Number of Calls Received (Mean)
4.09
4.80
5.35
5.79
7.21
4.34
2.92
5.56
4.71
4.74
4.73
0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00 5.50 6.00 6.50 7.00 7.50 8.00
55+
35-54
18-34
B.C.
Prairies
Ontario
Quebec
Atlantic
Women
Men
Overall
Mean
Number of Telephone Surveys Completed
46%
40%
47%
35%
35%
47%
51%
42%
40%
47%
44%
39%
44%
38%
46%
38%
40%
41%
38%
43%
39%
41%
13%
12%
12%
16%
20%
11%
8%
16%
14%
11%
13%
2%
3%
3%
4%
7%
2%
4%
3%
3%
3%
0% 20% 40% 60% 80% 100%
55+
35-54
18-34
B.C.
Prairies
Ontario
Quebec
Atlantic
Women
Men
Overall
Only This Call 2 to 4 5 to 10 11+
Number of Telephone Surveys Completed (Mean)
2.71
2.94
2.69
3.11
3.93
2.55
2.16
3.17
2.90
2.69
2.80
0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00
55+
35-54
18-34
B.C.
Prairies
Ontario
Quebec
Atlantic
Women
Men
Overall
Mean
Claimed Rate of Conversion
67%
60%
56%
56%
46%
60%
76%
54%
61%
60%
61%
5%
6%
5%
8%
10%
5%
2%
5%
6%
5%
5%
15%
19%
18%
17%
20%
19%
14%
18%
16%
18%
17%
13%
15%
22%
19%
24%
17%
8%
23%
16%
17%
17%
0% 20% 40% 60% 80% 100%
55+
35-54
18-34
B.C.
Prairies
Ontario
Quebec
Atlantic
Women
Men
Overall
All Calls 75%-99% 50%-74% <50%
Claimed Rate of Conversion (Mean)
0.82
0.79
0.74
0.77
0.71
0.78
0.88
0.74
0.80
0.78
0.79
0.00 0.25 0.50 0.75 1.00
55+
35-54
18-34
B.C.
Prairies
Ontario
Quebec
Atlantic
Women
Men
Overall
Mean
Observed Request : Conversion Ratio1.24
1.25
1.12
1.36
1.26
1.25 1.31
1.27
1.15
1.35
1.3
1.19
1.53
1.31
1.16
1.46
1.23
1.21
1.45
1.25
1.17
1.5
1.24
1.15
1.48
1.33
1.16
1.41
1.38
1.1
1
1.25
1.5
1.75
2A
TL
M 1
8-3
4A
TL
M 3
5-5
4A
TL
M 5
5+
AT
L W
18
-34
AT
L W
35
-54
AT
L W
55
+
QU
E M
18
-34
QU
E M
35
-54
QU
E M
55
+Q
UE
W 1
8-3
4Q
UE
W 3
5-5
4Q
UE
W 5
5+
ON
T M
18
-34
ON
T M
35
-54
ON
T M
55
+O
NT
W 1
8-3
4O
NT
W 3
5-5
4O
NT
W 5
5+
PR
M 1
8-3
4P
R M
35
-54
PR
M 5
5+
PR
W 1
8-3
4P
R W
35
-54
PR
W 5
5+
BC
M 1
8-3
4B
C M
35
-54
BC
M 5
5+
BC
W 1
8-3
4B
C W
35
-54
BC
W 5
5+
Observed Request : Conversion Ratio1.53
1.5
1.48
1.46
1.45
1.41
1.38
1.36
1.35
1.33
1.31
1.31
1.3
1.27
1.26
1.25
1.25
1.25
1.24
1.24
1.23
1.21
1.19
1.17
1.16
1.16
1.15
1.15
1.12
1.1
1
1.25
1.5
1.75O
NT
M 1
8-3
4P
R W
18
-34
BC
M 1
8-3
4O
NT
W 1
8-3
4P
R M
18
-34
BC
W 1
8-3
4B
C W
35
-54
AT
L W
18
-34
QU
E W
18
-34
BC
M 3
5-5
4Q
UE
M 1
8-3
4O
NT
M 3
5-5
4Q
UE
W 3
5-5
4Q
UE
M 3
5-5
4A
TL
W 3
5-5
4A
TL
M 3
5-5
4A
TL
W 5
5+
PR
M 3
5-5
4A
TL
M 1
8-3
4P
R W
35
-54
ON
T W
35
-54
ON
T W
55
+Q
UE
W 5
5+
PR
M 5
5+
ON
T M
55
+B
C M
55
+Q
UE
M 5
5+
PR
W 5
5+
AT
L M
55
+B
C W
55
+
Claimed Request : Conversion Ratio1.85
1.51
1.231.27
1.23 1.31
1.19
1.15
1.11.2
1.13
1.08
1.46
1.22
1.23
1.26 1.36
1.19
1.66
1.36
1.36
1.68
1.241.38
1.19 1.251.38
1.31 1.41
1.29
1
1.25
1.5
1.75
2A
TL
M 1
8-3
4A
TL
M 3
5-5
4A
TL
M 5
5+
AT
L W
18
-34
AT
L W
35
-54
AT
L W
55
+
QU
E M
18
-34
QU
E M
35
-54
QU
E M
55
+Q
UE
W 1
8-3
4Q
UE
W 3
5-5
4Q
UE
W 5
5+
ON
T M
18
-34
ON
T M
35
-54
ON
T M
55
+O
NT
W 1
8-3
4O
NT
W 3
5-5
4O
NT
W 5
5+
PR
M 1
8-3
4P
R M
35
-54
PR
M 5
5+
PR
W 1
8-3
4P
R W
35
-54
PR
W 5
5+
BC
M 1
8-3
4B
C M
35
-54
BC
M 5
5+
BC
W 1
8-3
4B
C W
35
-54
BC
W 5
5+
Claimed Request : Conversion Ratio1.85
1.68
1.66
1.51
1.46
1.41
1.38
1.38
1.36
1.36
1.36
1.31
1.31
1.29
1.27
1.26
1.25
1.24
1.23
1.23
1.23
1.22
1.20
1.19
1.19
1.19
1.15
1.13
1.10
1.08
1.00
1.25
1.50
1.75
2.00A
TL
M 1
8-3
4P
R W
18
-34
PR
M 1
8-3
4A
TL
M 3
5-5
4O
NT
M 1
8-3
4B
C W
35
-54
BC
M 5
5+
PR
W 5
5+
PR
M 3
5-5
4O
NT
W 3
5-5
4P
R M
55
+B
C W
18
-34
AT
L W
55
+B
C W
55
+A
TL
W 1
8-3
4O
NT
W 1
8-3
4B
C M
35
-54
PR
W 3
5-5
4A
TL
M 5
5+
ON
T M
55
+A
TL
W 3
5-5
4O
NT
M 3
5-5
4Q
UE
W 1
8-3
4B
C M
18
-34
ON
T W
55
+Q
UE
M 1
8-3
4Q
UE
M 3
5-5
4Q
UE
W 3
5-5
4Q
UE
M 5
5+
QU
E W
55
+
Expected Future Trend
17%
14%
15%
12%
15%
17%
17%
11%
16%
56%
63%
65%
68%
64%
65%
49%
64%
61%
19%
19%
17%
15%
18%
12%
29%
18%
18%
8%
4%
2%
5%
3%
5%
5%
7%
5%
0% 20% 40% 60% 80% 100%
55+
35-54
18-34
B.C.
Prairies
Ontario
Quebec
Atlantic
Overall
Less Same More DK/NR
In the next 12 months, do you think you will be more likely, less likely, or just as likely to complete a phone survey, if asked??
9%
13%
17%
14%
7%
19%
15%
16%
16%
73%
64%
57%
67%
65%
50%
63%
59%
61%
16%
18%
21%
15%
23%
23%
17%
19%
18%
2%
5%
5%
4%
5%
8%
4%
5%
5%
0% 20% 40% 60% 80% 100%
>$100k
$60k-$100k
<$60k
Univ/Coll
Some PS
HS or Less
Female
Male
Overall
Less Same More DK/NR
Who has done any sort of online survey in past year?
25%
40%
45%
41%
44%
37%
29%
33%
36%
43%
51%
51%
46%
44%
49%
50%
53%
48%
31%
10%
5%
13%
12%
14%
21%
14%
15%
0% 20% 40% 60% 80% 100%
55+
35-54
18-34
B.C.
Prairies
Ontario
Quebec
Atlantic
Overall
Did at least 1 Online, but did none Offline
In the past 12 months, that is since May 2009, how many times have you agreed to participate in any sort of online survey conducted over the Internet?
53%
47%
25%
44%
18%
18%
37%
36%
36%
45%
48%
51%
43%
48%
48%
45%
52%
48%
2%
5%
25%
13%
34%
34%
18%
12%
15%
0% 20% 40% 60% 80% 100%
>$100k
$60k-$100k
<$60k
Univ/Coll
Some PS
HS or Less
Female
Male
Overall
Did at least 1 Online, but did none Offline
Who has joined online panels?
8%
16%
18%
17%
15%
14%
12%
14%
14%
62%
74%
77%
71%
73%
72%
67%
72%
71%
31%
9%
5%
12%
12%
14%
21%
14%
15%
0% 20% 40% 60% 80% 100%
55+
35-54
18-34
B.C.
Prairies
Ontario
Quebec
Atlantic
Overall
Joined Online, but Never Joined Offline
Have you ever joined an online group, community or panel for the purpose of completing surveys over the Internet?
18%
16%
12%
17%
22%
8%
16%
12%
14%
79%
79%
64%
76%
66%
58%
66%
76%
71%
2%
5%
24%
8%
12%
34%
18%
12%
15%
0% 20% 40% 60% 80% 100%
>$100k
$60k-$100k
<$60k
Univ/Coll
Some PS
HS or Less
Female
Male
Overall
Joined Online, but Never Joined Offline
Number of Panels Joined
© Harris Interactive
Expected Future Trends
23%
16%
65%
61%
11%
18%
1%
5%
0% 20% 40% 60% 80% 100%
Online panelists
Telephone
Less Same More DK/NR
More questions…• The toughest strata to reach by phone are among the most participatory
online.– Is dove-tailing samples a way to help, or is it more risk than benefit?
• We build/weight both phone and online samples to look like Census profiles.– If we know that either of the two are not truly identical to the Census
profile and vary in identifiable ways, would we design samples to reflect these limitations?
• Phone response rates have continued to decline, but perhaps that decline has slowed. Our findings show as online panels grow, response rates also decline.– Should we read anything into the responses on respondents’
expectations for the future?