Biting the hand and bending the rules

By Annie Pettit PhD, Chief Research Officer, Conversition

Best sentiment

system ever!

Our sentiment

is

99% valid

Misinformation #1

4

Real-life Data Analysis: How Many Licks to the Tootsie Roll Center of a Tootsie Pop? by Cory Heid, guest blogger Almost all of us have tried a Tootsie Pop at some point. I’m willing to bet that most of us also thought, “I wonder how many licks it does take to get to the center of the Tootsie Pop?” If you haven’t wondered about this, here’s the classic commercial that may get you more curious. Personally, I was not very satisfied with the owl's answer of “3,” so I decided to continue the little boy’s quest to find the number of licks required to reach the center of a Tootsie Pop. Research Looking around the ‘net, I found that other studies done by student researchers at various universities have reached very different answers. These students, who represented some significant research institutions and engineering schools, used different licking methodologies and/or licking machines and licking experiments. As I mentioned, the results varied greatly. See for yourself: Since I lacked both the equipment and desire to build a licking machine, a simpler licking experiment seemed appropriate. I wanted to assess which of these factors would have the most impact on the effectiveness of the Tootsie-Pop licking: Force of the lick Temperature of the licker's mouth pH Level of the licker's saliva Solubility of the licker's saliva Lab Work Once I selected my factors, I incorporated them into an experiment. I chose simulate the different levels of each factor in the lab. For my force simulation, I placed my Tootsie Pop in a beaker with 150 mL of water and a magnetic stirrer. I placed the beaker on a hot plate that could spin the stirrer and create a circular motion in the water. I then pulled the Tootsie Pop out of the water every minute and measured the height and width of the pop on the thicker band around the pop. I did this each minute until the pop revealed a noticeable amount of chocolate (the elusive ‘center’). This method had some issues. The chocolate-flavored Tootsie Pops made the water very murky, so I omitted them from the lab testing. I also could not track the speed of the circulating water. Instead, I used the speed dial’s indicator on the magnetic stirrer (1 being slowest speed, 10 being fastest speed). After several tests at speeds 1, 2, 4 and 6, I was able to gather an ample amount of data to perform some analysis. I then moved on to the temperature tests. Like the force tests, these used a beaker, 150 mL of water, and a hot plate. I heated the water to different temperatures and measured the height and width of the Tootsie Pop every minute until reaching the center. The next test involved pH. I used a beaker, and 150 mL of water as the basis of my tests. I added a small amount of either baking soda (to make the solution more basic) or vinegar (to make the solution more acidic). I then recorded and measured the height and width of the pop at regular intervals. My final test, for solubility, consisted of putting the Tootsie Pop in a very large container containing a gallon of water. I dipped the pop in the water for one minute and took height and width measurements. In addition, I stirred the water to make sure it was diluted as possible and evenly distributed. My goal was to have enough data to analyze to help determine important factors in Tootsie Pop decay.

Long Verbatims have poor validity

5

• I want Cracker Jacks.

• Cracker Jacks are the shit

• i want Cracker Jacks soo bad

• I am addicted to Cracker Jacks...

Short verbatims have high validity

6

Which sentiment trend is more valid?

7

5= Positive

1= Negative

3.1

3.2

3.3

3.4

Short Long

“The best academic study I've seen showed only 82% agreement between two human sentiment annotators, rising to 90% if uncertain cases are

removed.“

Seth Grimes, June 2011,

http://www.b-eye-network.com/view/15276

The Truth

8

Sentiment

scoring is one

size fits all

Misinformation #2

9

One size scoring or custom scoring?

10

10%

20%

30%

40%

50%

60%

Jan Feb Mar Apr May Jun Jul Aug Sep

OneSize Scoring Custom Scoring

Standard scoring or category scoring?

11

20%

40%

60%

80%

Standard Category

Twitter

(survey)

is a Godsend

Misinformation #3

12

0%

20%

40%

60%

Bottom 2 Boxes Neutral Top 2 Boxes

0%

20%

40%

60%

Bottom 2 Boxes Neutral Top 2 Boxes

0%

20%

40%

60%

Bottom 2 Boxes Neutral Top 2 Boxes

0%

20%

40%

60%

Bottom 2 Boxes Neutral Top 2 Boxes

Guess the brands (Tootsie Roll, Chicago Bears,

Cracker Jack, Ferris Wheels)

13

Blogspot

Tumblr

Twitter

Facebook

I can measure

your brand

awareness

Misinformation #4

14

Pfizer has greater awareness than Unilever

15

0

10000

20000

30000

Pfizer Unilever

P90X has even greater awareness than Pfizer

16

0

20000

40000

60000

80000

P90X Pfizer Unilever Red Rose Tea

Cracker Jacks and Tootsie Rolls are here

Demographics

are available

Misinformation #5

17

Care to Crosstab Demos on Starbucks Data?

18

Total MaleFemal

eUS

Europe

AsiaCanad

aMiddle East

Pacific Africa 25-34 18-24 35-44 45-54 55+

Sample Size 18557 4182 3706 2118 358 247 220 47 35 20 12 9 6 6 2

0

5000

10000

15000

20000

Social media

results are

instantaneous

Misinformation #6

19

Antacid: Fast or Good?

20

0%

20%

40%

60%

Negative Neutral Positive

0%

20%

40%

60%

80%

Negative Neutral Positive

Fast: With Spam Good: Without Spam

What is GMC?

21

GMC: Fast or Good?

22

0%

25%

50%

75%

100%

GMC Auto GMC Any

B2B N T2B

Insight is

instant

Misinformation #7

23

Expertise is required

24

+ =

The internet

was made for

engagement

Misinformation #8

25

Famous people signed up for this

26

27

28

29

• http://www.mra-net.org/rq/documents/MRA_IMRO_SMR16.pdf

• http://www.casro.org/pdfs/1011/Social_Media_Research_Guidelines.pdf

• http://www.esomar.org/uploads/public/knowledge-and-standards/codes-and-guidelines/ESOMAR_Guideline-on-Social-Media.pdf

• http://www.mrs.org.uk/pdf/2012-04-04%20Online%20data%20collection%20and%20privacy.pdf

MRX guidelines say don’t do that

Who’s a member?

31

Why aren’t listening companies members?

32

Stifles my creativity

#1 Why not?

33

I don’t want someone telling

me what to do

#2 Why not?

34

They will put me out of

business

#3 Why not?

35

Confessions

•Strengths and weaknesses

•Sentiment analysis is nowhere near perfect

•Speed will not result in high quality

•Demographics are weak

•Stance on privacy

What to look for

36

Thank you!

37