How people talk about us behind our backs

Tweet! #WAPOR @LoveStats

How people

talk about us behind our backs

Annie Pettit, Chief Research Officer


Agenda

•Are we being good citizens?

•How are people reacting to us?

•Can we fix it?


Method• Social Media Data

– Hypothesis generation– N>200 000– Global English

• Survey Data– Hypothesis confirmation and precision– N=400– Non-probability incentivized panel, Sampled and

Weighted to US Census Rep Not because the target group is census rep but because we’re familiar with census rep


Survey Instructions• We are really interested in your honest answers. Don’t worry about hurting our feelings or speaking a scary truth. You will not be penalized for sharing any of your opinions. We need to hear them!

• (Please be honest. We will not penalize your account.)

• (Please don't be scared to tell the truth. We will not penalize your account.)

• (Please, please, tell us the truth. :)


Trust


Who to trust?• Do ya think these paid survey thing is a scam?

• @aaa took an online survey once. Seemed legit

• i just took a survey.... is it a scam or is it just me.

• hey all pls confirm me is this trust worthy

• @bbb surveys are scams usually

• @ccc gallop survey totally fraud

• is opinion outpost trust worthy?

• I dont trust ipsos synovate..at all.


It’s not spam if…

• Harder to convince: Poor responders, Men, Older people• Younger people trust their friends, social media comments


Remind People•You’re not selling anything

– Particularly on purchase intent questions– “We’re not asking you to buy something. We’re just curious if you have any

plans.”

•You don’t want their credit card info•You don’t need them to download anything

•Your registration, certification credentials


Survey Length


Survey Length• this a long survey man smh

• That's one long ass survey

• wow that was a long survey

• And another long time bump!

• This survey is sooo long >.<

• Taking this long ass survey.

• This Boring Ass Long Survey..

• Gotta do this long ass survey

• This survey is to damn long ..

• Thisss SURVEY long Ass Hell Man !

• Damnn that survey was long as shit

• Omg that survey thing was long af'

• Damnn that survey was long as shit

• online apps with long ass surveys<<

• Taking so long to do this survey ugh

• I got to do these 2 long ass surveys

• Filling out this long ass survey. <<

• Taking so long to do this survey ugh

• Mothafn' survey was long as shit.

• This survey is really really long lol

• Filled out that long ass survey.... :/

• This survey is gonna take a long while

• Dang I hate doing them long ahh surveys


It’ll Only Take Five Minutes• SSI survey about mobile devices. The survey says 1 mins, but it will take much longer than that. Also, survey is from "qualtrics", so don't speeding through the survey or else you won't get credit. btw, another purple 2 cents ad is up

• I got an email survey that said it would take 1 min so I did it. It actually took about 20 - they made me watch a stupid video about Audi and Le Mans.

• hate surveys that say "this will only take 2 minutes" and end up taking 15 because i like to write detailed feedback


12 Minute Surveys

•Shorter surveys preferred by women, older people


Lying


You Should Lie• Oh my god, I made a male account & started seeing

shitloads of surveys compared shoulda done that sooner :/

• Just lie on the surveys. Pretend you are a middle aged Hispanic woman. I made 18,000 in three months.

• Oh hello telephone survey people, why yes my name is monty and im a 28yr old commercial airline pilot with a phd. Lol

• I love to answer polls, but I tend to lie on my responses.

• Do I lie on this survey or do I be truthful? I dont wanna incriminate my self...


27% Confess to Lying*

*Not confessing does not equal innocence


I Lied Because…

Answer wasn’t there

Trying to qualify

In a rush

Tired Having fun

Randomly

clicking

Upset with hosting

Don’t know


I Confess Also…

• Partials: Do not read, Speed, Distracted, Answer on behalf• Poor responders: Don’t admit distraction, Admit to random

clicking


Always give an out•None of the above•Never•Less than X•More than Y•Other•Please specify: _____________


Transparency


Why demographics?•Everytime a survey is done they always ask if ure hispanic like why does it fn matter! #whooocares

•Why do surveys ask your race and include the option "other"? What the f could other be? "Me? I'm actually a bellydancing cactus."


Why Screening?•Survey just asked me where am I living, and screened me out immediately when I said East Midlands

•I was doing a survey for the trailer and it looks boring AF. I gave my honest opinion and because it was all negative they screened me out.


Screen Too Late•Just got 10 mins through a 15 min survey only to be disqualified.... Ggrrrrrrr

•Got a survey on films, DON'T COMPLETE IT! I got to end and screened me out, losing 20 mins of time Survey# 266616

•Fed up of doing 15 minutes of a survey then getting screened out, why can't they ask the relevant questions at the beginning


I don’t understand you

•Partials do NOT understand•Poor responders do NOT understand


Demographics•Useless

– “To classify responses”– “To categorize the data”

•Useful– “For example, to understand how men

and women, or older and younger people differ

– “To ensure we listen to many different types of people”


Screening•Screen people out ASAP

•Take the extra minute and tell people why– Those are all the questions we have

for you today.– We are sorry to screen you out but we

have already heard from a number of people with your characteristics and need to make sure we also listen to people with other characteristics


Screening•Let people know they may be screened because they DO use or product or because they do NOT use a product

•Enforce recruitment and invitation caps so that certain groups don’t get far more surveys than other groups

–You know that women get fewer invites so recruit less of them


Incentives Pay for Boredom


They EARN Incentives• ugh @ this boring ass survey.• i hate filling these surveys in..boring• Focus group today good money but so boringggg• swagbucks is mindnumbingly boring got me 9 bucks lol• Done with that Youth Risk Behavior Survey -_- shit boring!!• online surveys are intensely boring but I want the Paypal

points...• Just participated in a focus group. Boring but LOVE the

incentives! $$• Tonight was the most boring 'live' market research I have

ever done!!!!• UGh this survey is so boring but I'm making ten dollars an

hour so its kind of nice


TopicsLove

Hate


Money Ranks Highest

•Partials rank every incentive higher

1

3

5

7

9

Top Rank

Bottom Rank


Don’t Be Boring•Loosen up your grammar

– “What is the best idea you can think of?”

•Use more casual language– “Tell me why you like it.”

•Try a light joke or two– “May the survey force be with you”


Responder Thoughts• I absolutely loved this survey! Thank you for wanting my

opinion. :)• I have enjoyed taking this survey, because it tells me that

some company actually cares about improving its surveys. • I have to admit I was wondering if any company, was or

would ever pay attention to panelists concerns usually it always about profit declared or not, but it's true.. But it's comforting and showing some good faith to me, being considered and that this company has seemed to notice these concerns need to be addressed.. Thank you.

• I'm glad to see that you might have some concerns about people not being completely truthful with their answers. I always answer the questions truthfully. However, sometimes the questions are asked in such a manner that none of the above really apply.


Conclusions• We think we are being transparent but we are NOT

• We can be honest without betraying the method

• Must write better• Must not interpret data as perfect


Thank you!

Annie PettitChief Research OfficerPeanut [email protected]@LoveStats on Twitterwww.peanutlabs.com

: )