Global Patterns in Panel Research

BySteven Gittelman, Ph.D.

and Elaine Trimarchi

Methods

Compared 17 U.S. panels and 25 global panels

Selected demographic quotas (age, income, gender, ethnicity) were used to simulate census.  

Median length was 15 minutes.

Questions covered:  Technology and the media, Participation in market research, Buyer Behavior, Values and lifestyle, Demographics, Questionnaire Satisfaction.

Respondent Types Professional Respondents fall into four categories:

• (1) Self report taking on‐line Surveys “practically every day”.

• (2) Self report (open ended) taking over 30 online surveys “in the past month”.

• (3)  Multiple panel membership > 5 panels.

• (4) Respondent panel tenure. 

Percent Respondents Doing More than 30 Surveys/Month

Red = US Panels Green = International Panels

0%

5%

10%

15%

20%

25%

30%

35%

40%

Singap

ore

Portugal

Hong K

ong

Switzerl

and

Finlan

d

Czech

Rep

Ukraine

Russia

Brazil

Argen

tina

ChinaS. K

orea

Norway

Denmark

Poland

Taiw

anUS11US10

Sweden

Fran

ceSpainIta

lyGerm

any

Canad

aUS16 UK

Australi

aJa

pan

US12US9US6

US17US14US13US7US18US8

Perc

ent 3

0+ s

urve

ys p

er m

onth

US11=RiverUS10= Social Network

Percent Respondents Enrolled in > 4 Panels

0%

10%

20%

30%

40%

50%

60%

70%

Singap

ore

Portugal

Hong K

ong

Switzerl

and

Finlan

d

Czech

Rep

Ukraine

Russia

Brazil

Argen

tina

ChinaS. K

orea

Norway

Denmark

Poland

Taiw

anUS11US10

Sweden

Fran

ceSpainIta

lyGerm

any

Canad

aUS16 UK

Australi

aJa

pan

US12US9US6

US17US14US13US7US18US8

Enro

lled

in o

ver 5

Pan

els

or M

ore/

mon

th

Max Age on Panel by Panel in the U.S.

0%

20%

40%

60%

80%

100%

M6 M7 M8 M9 M10 M11 M12 M13 M14 M16 M17 M18 GrandTotal

Panel

% o

f Res

pond

ents

0 Months 6 Months 12 Months 18 Months 24 Months

US Sociographic segment distribution by panel and phone.

0%

20%

40%

60%

80%

100%

US1*US2*US3*US4*US5* US6

US7US8US9US10US11US12US13US14US16US17US18

US Pho

ne*

% o

f Res

pond

ents

in S

egm

ent

High Computer/Stays Informed Happy with Life/Not Computer Opinionated/Not Computer

* EM Algorithm for Missing Data & Logit Model for Segmentation

Social Network

US and Global Distribution of Buyer Behavior among Panels

0%

20%

40%

60%

80%

100%

US6US7US8US9US10US11US12US13US14US16US17US18 UKSpain

Germany

FranceIta

lyJapan

Total US Panels

Perc

enta

ge o

f Res

pond

ents

Broad Range/Credit Price Sensitive Shoppers Credit/Environment Domestic/Coupons

Social Network

-10-8-6-4-202468

10

M11 M3 M4 M10 M2 M16 M15 M1 M9 M12 M7 M13 M5 M14 M6 M17 M8 M18

Stan

dard

Err

ors

Conventional Purlchasers On-liners Low Card OL Banking

97%99%

River SocialNetwork

PointSystem

UK Access Panels

Statistical Panel Profiles Against Buyer Segments

The variability between panels is a problem that requires a solution.

• Blending is the solution.• The database is the source of that solution.

• The “Grand Mean” is a new platform for stability.

• Optimization is the road map.

Optimization Profile

0.00

0.01

0.02

0.03

0.04

0.050.06

0.07

RM

S Er

ror

0%20%

40%60%

80%100%0% 20% 40% 60% 80% 100%

Percent of M8

Percent of M17

+/-14.8%

+/-10.8%

+/-12.9% +/-14.9%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Broad Range/ Credit Price SensitiveShoppers

Credit/ Environment Domestic/ Coupons

Perc

ent o

f Res

pond

ents

Buyer Behavior Segments

Expected Range of Values for a Random 3 Panel Sample Showing 1.281 Standard Errors (20% of being beyond this range) in the 

U.S.

+/- Coefficient of variation

+/-20.8%+/-15% +/-17.1%

0%

10%

20%

30%

40%

50%

60%

High Computer/ StaysInformed

Happy with Life/ NotComputer

Opinionated/ Not Computer

Perc

ent o

f Res

pond

ents

Sociographic Segments

Expected Range of Values for a Random 3 Panel Sample Showing 1.281 Standard Errors (20% of being beyond this range) in the 

U.S.

+/- Coefficient of Variation

+/- 22.1%

+/- 18.6% +/- 15.8% +/- 16.4%

0%

10%

20%

30%

40%

50%

60%

Internet Stay Informed Enjoys Politics Concerned

Perc

ent o

f Res

pond

ents

Media Segments

Expected Range of Values for a Random 3 Panel Sample Showing 1.281 Standard Errors (20% of being beyond this range) in the 

U.S.

+/- Coefficient of Variation

We can optimize to the Grand Mean.In this example we show the expected standard error from 

the Grand Mean based on the average of all random choices (8.31%). 

Based on equal weighting of three panels selected by optimization to the Grand Mean (2.36%)

… and the same three panels blended in proportions to optimize to the Grand Mean (0.40%).

Panels Optimum AverageExpected (1 SE) Inherent (1 SE)

M8 24% 33%M17 26% 33%M12 50% 34%

Root Mean Square Error 0.40% 2.36% 8.31% 2.45%

Two more examples…but here the panel selection is random and not optimized.  Only the weightings are 

optimized to the Grand Mean.

Panels Optimum AverageExpected (1 SE) Inherent (1 SE)

M8 0% 33%M13 91% 33%M16 9% 34%

Root Mean Square Error 3.6% 7.8% 8.3% 2.4%

Panels Optimum AverageExpected (1 SE) Inherent (1 SE)

M10 8% 33%M13 66% 33%M16 27% 34%

Root Mean Square Error 1.6% 12.3% 8.3% 2.4%

Thank youSteven Gittelman, Ph.D.

and Elaine Trimarchi 200 Carleton Avenue

East Islip, New York 117301‐631‐277‐7000