Cooperative Respondents, Enlightened Clients, and Other Research Myths

  • Published on
    03-Jan-2016

  • View
    33

  • Download
    4

DESCRIPTION

Cooperative Respondents, Enlightened Clients, and Other Research Myths. CMAG, March 13 th , 2008. Respondent Cooperation (or more specifically, lack there of). CASRO 10/02 Respondent Cooperation Summit 11/06 Quirks 1/07 CMAG Challenge 11/07. - PowerPoint PPT Presentation

Transcript

  • Cooperative Respondents,Enlightened Clients,and Other Research MythsCMAG, March 13th, 2008

  • Respondent Cooperation(or more specifically, lack there of)CASRO 10/02 Respondent Cooperation Summit 11/06Quirks 1/07

    CMAG Challenge 11/07

    We get that theres a respondent cooperation issue. Dont just revisit the problem. Help us find some kind of a solution?

  • Great Ideas We Wont Discuss Improved Sampling Procedures Better IncentivesProprietary PanelsReinstituting Non-Response WorkMore Engaging Administration

  • A Novel Thought Can We Create a Win for Respondents by:encouraging their cooperation with more engaging surveys?

    Might This Also Create a Win for Clients:because more engaging surveys deliver more enlightening results?

  • One Ecosystem Three SpeciesClients, respondents, and the research community are inextricably entwined. A win/win model will benefit both clients and respondents but were the biggest winners.

    It is therefore not surprising that the problem is ours to resolve. Respondents the supply of informationClients the demand for informationMarket Research - the brokers of information

  • RespondentsHow Do We Abuse Them? The Seven Deadly Sins of Market Research Design

    Client Directed MethodologyLack of Focus (Omnibus Studies)Inappropriate TargetingInsulting IntelligenceInept ConstructionExposed Game Brutal Length

  • Applying the Groucho Marx testRecognizing their priorities regarding surveysUnderstanding that they like to pleaseProviding interesting constructionDelivering tolerable length surveysVolunteering as Guinea Pig #1

    RespondentsHow Might We Do Better?

  • Folks who:need to know a lot tend to overestimate what they know about their businessoften overestimate what they know about market researchare native leaders

    ABOVE ALL:Clients are decision makers

    ClientsWho are They?

  • Clients want insight not just information. Dont drown me in data just tell me what I need to know. Delivering such insight is an easy promise to make but not so easy to keep. ClientsWhat Do They Seek?

  • Insight, like beauty, is in the eye of the beholder and the beholder is your client.Not new?Not needed?Not insight!

    So What is Insight?And the first fundamental requirement for generating Client Insight is?

    Cooperative Respondents!

  • Designing to Insight Clientswithout Inciting RespondentsEver been asked any of these questions?

    How come my tracker seldom gives me anything new or useful?What actions can I take with the tracker feedback regarding price perceptions?Speaking of price, is there any reliable methodology that can help me price my product? How about ways to better evaluate programs?

    In the time remaining, lets see how many of these issues we can address with approaches meeting our new specifications:

    Considerate of the respondent yet Insightful for the client.

  • Conventional ShortcomingsMore respect for longitudinal consistency than current relevancyLimited argument for attribute refreshing Measurement calcification (MBOs)Surveys are brutally long and generally dullNOTHING CHANGESAnd lots more

    Relevant Space Methodology Brand Tracking System A Win/Win Example

  • RELEVANTSPACE:Providing a handful of ratings on brands you know and issues you care about. Respondent Respect Relevant Space Methodology

  • Client InsightKnowing What MattersInsights

    Knowledgeable respondents know what mattersExperiential attributes play key role in developed marketsEmotive attributes serve as surrogate for experience elsewhereThe less you know, the more that brand mattersDeveloped markets can guide emerging markets

    Chart1

    75.448.7

    68.241.4

    6643.4

    65.845.1

    58.238.9

    57.449.7

    52.542.7

    50.939.8

    46.539.3

    45.138.5

    4331

    37.938.7

    3741.3

    36.633.1

    31.430.4

    30.830.6

    25.927

    21.431.7

    20.931.1

    20.426.9

    18.539.6

    17.828.4

    15.728.6

    14.632.2

    14.223.7

    9.826.1

    9.840.9

    8.331.4

    Mature Market Emerging Market

    Expertise Predominant

    Emotive Predominant

    Expertise Predominant

    Expertise Predominant

    EasePredominant

    NA/UK/France/Germany

    China/India

    Sheet1

    North AmericaUKJapanItalyIndiaGermanyFranceChina

    Estimated Share of Future Networking Budget to Cisco19.2018.909.8018.9016.8020.5021.707.20

    ImportanceCisco SystemsCisco SystemsCisco SystemsCisco SystemsCisco SystemsCisco SystemsCisco SystemsCisco Systems

    Scalable32.15.905.925.305.416.185.785.815.42

    Easily Upgradeable38.55.585.354.515.516.315.205.645.46

    Manageable44.96.115.724.695.126.335.465.425.35

    Low Maintenance Cost37.45.044.593.734.505.964.124.325.01

    Expert Maintenance51.75.905.604.875.506.245.485.945.18

    Visionary15.75.875.714.995.956.256.005.444.93

    Accommodating Personnel24.65.435.204.255.186.405.225.155.59

    Network of Partners19.85.935.405.065.466.225.625.945.13

    Low Priced Products27.34.663.863.504.365.653.814.154.85

    Reliable65.86.105.995.535.816.366.056.195.37

    Recognized Brand17.86.336.225.395.696.556.356.245.45

    Easy to Use39.85.004.914.675.056.274.685.125.25

    High Performance48.16.165.985.545.736.526.056.095.31

    Disaster Recovery46.45.805.685.015.296.205.465.215.19

    Robust Security59.36.116.015.385.936.225.956.095.69

    Comprehensive Training & Cert.17.35.545.525.365.596.205.695.565.88

    Highly Compatible47.35.965.535.285.486.285.435.605.35

    Understands Business23.35.564.933.915.256.295.165.275.26

    Financially Sound28.86.146.105.185.586.366.005.995.43

    Well-Respected22.26.236.264.975.736.566.216.195.61

    Socially Responsible14.85.694.605.335.306.264.615.415.56

    Consistently Delivers56.15.755.504.775.546.105.345.535.47

    Responds Quickly58.65.505.254.585.156.275.085.285.21

    Warranty Can Count On57.95.945.705.015.606.435.696.005.53

    Sets Critical Standards25.86.086.105.385.696.316.106.065.44

    Invests in R&D206.136.085.155.746.306.086.015.61

    Understands Your Network Environ.39.75.855.513.965.326.365.495.555.05

    Pioneers Innovative Features18.86.145.844.765.756.115.885.845.35

    Likelihood to recommend4.294.053.494.064.584.154.073.77

    NAUKJapanItalyIndiaGermanyFranceChina

    Estimated Share of Future Networking Budget to Cisco19.218.99.818.916.820.521.77.2

    Importance

    Scalable32.15.905.925.305.416.185.785.815.42

    Easily Upgradeable38.55.585.354.515.516.315.205.645.46

    Manageable44.96.115.724.695.126.335.465.425.35

    Low Maintenance Cost37.45.044.593.734.505.964.124.325.01

    Expert Maintenance51.75.905.604.875.506.245.485.945.18

    Visionary15.75.875.714.995.956.256.005.444.93

    Accommodating Personnel24.65.435.204.255.186.405.225.155.59

    Network of Partners19.85.935.405.065.466.225.625.945.13

    Low Priced Products27.34.663.863.504.365.653.814.154.85

    Reliable65.86.105.995.535.816.366.056.195.37

    Recognized Brand17.86.336.225.395.696.556.356.245.45

    Easy to Use39.85.004.914.675.056.274.685.125.25

    High Performance48.16.165.985.545.736.526.056.095.31

    Disaster Recovery46.45.805.685.015.296.205.465.215.19

    Robust Security59.36.116.015.385.936.225.956.095.69

    Comprehensive Training & Cert.17.35.545.525.365.596.205.695.565.88

    Highly Compatible47.35.965.535.285.486.285.435.605.35

    Understands Business23.35.564.933.915.256.295.165.275.26

    Financially Sound28.86.146.105.185.586.366.005.995.43

    Well-Respected22.26.236.264.975.736.566.216.195.61

    Socially Responsible14.85.694.605.335.306.264.615.415.56

    Consistently Delivers56.15.755.504.775.546.105.345.535.47

    Responds Quickly58.65.505.254.585.156.275.085.285.21

    Warranty Can Count On57.95.945.705.015.606.435.696.005.53

    Sets Critical Standards25.86.086.105.385.696.316.106.065.44

    Invests in R&D206.136.085.155.746.306.086.015.61

    Understands Your Network Environ.39.75.855.513.965.326.365.495.555.05

    Pioneers Innovative Features18.86.145.844.765.756.115.885.845.35

    Likelihood to recommend4.294.053.494.064.584.154.073.77

    ImportanceFranceGermanyNAUKItalyIndiaJapanChina

    Estimated Share of Future Networking Budget to Cisco21.720.519.218.918.916.89.87.2

    Reliable666.196.056.105.995.816.365.535.37

    Robust Security596.095.956.116.015.936.225.385.69

    Responds Quickly595.285.085.505.255.156.274.585.21

    Warranty Can Count On586.005.695.945.705.606.435.015.53

    Consistently Delivers565.535.345.755.505.546.104.775.47

    Expert Maintenance525.945.485.905.605.506.244.875.18

    High Performance486.096.056.165.985.736.525.545.31

    Highly Compatible475.605.435.965.535.486.285.285.35

    Disaster Recovery465.215.465.805.685.296.205.015.19

    Manageable455.425.466.115.725.126.334.695.35

    Easy to Use405.124.685.004.915.056.274.675.25

    Understands Your Network Environ.405.555.495.855.515.326.363.965.05

    Easily Upgradeable395.645.205.585.355.516.314.515.46

    Low Maintenance Cost374.324.125.044.594.505.963.735.01

    Scalable325.815.785.905.925.416.185.305.42

    Financially Sound295.996.006.146.105.586.365.185.43

    Low Priced Products274.153.814.663.864.365.653.504.85

    Sets Critical Standards266.066.106.086.105.696.315.385.44

    Accommodating Personnel255.155.225.435.205.186.404.255.59

    Understands Business235.275.165.564.935.256.293.915.26

    Well-Respected226.196.216.236.265.736.564.975.61

    Invests in R&D206.016.086.136.085.746.305.155.61

    Network of Partners205.945.625.935.405.466.225.065.13

    Pioneers Innovative Features195.845.886.145.845.756.114.765.35

    Recognized Brand186.246.356.336.225.696.555.395.45

    Comprehensive Training & Cert.175.565.695.545.525.596.205.365.88

    Visionary165.446.005.875.715.956.254.994.93

    Socially Responsible155.414.615.694.605.306.265.335.56

    Likelihood to recommend4.074.154.294.054.064.583.493.77

    FranceGermanyNAUKItalyIndiaJapanChina

    Estimated Share of Future Networking Budget to Cisco21.720.519.218.918.916.89.87.2

    Importance

    1Reliable66

    2Robust Security59

    3Responds Quickly59

    4Warranty Can Count On58

    5Consistently Delivers56

    6Expert Maintenance52

    7High Performance48

    8Highly Compatible47

    9Disaster Recovery46

    10Manageable45

    11Easy to Use40

    12Understands Your Network Environ.40

    FranceGermanyNAUKItalyIndiaJapanChina

    Estimated Share of Future Networking Budget to Cisco21.720.519.218.918.916.89.87.2

    Importance

    13Easily Upgradeable39

    14Low Maintenance Cost37

    15Scalable32

    16Financially Sound29

    17Low Priced Products27

    18Sets Critical Standards26

    19Accommodating Personnel25

    20Understands Business23

    21Well-Respected22

    22Invests in R&D20

    23Network of Partners20

    24Pioneers Innovative Features19

    25Recognized Brand18

    26Comprehensive Training & Cert.17

    27Visionary16

    28Socially Responsible15

    Likelihood to recommend

    Sheet2

    FranceGermanyNAUKItalyIndiaJapanChina

    21.720.519.218.918.916.89.87.2

    Importance

    Reliable66

    Robust Security59

    Responds Quickly59

    Warranty Can Count On58

    Consistently Delivers56

    Expert Maintenance52

    High Performance48

    Highly Compatible47

    Disaster Recovery46

    Manageable45

    Easy to Use40

    Understands Your Network Environ.40

    Easily Upgradeable39

    Low Maintenance Cost37

    Scalable32

    Financially Sound29

    Low Priced Products27

    Sets Critical Standards26

    Accommodating Personnel25

    Understands Business23

    Well-Respected22

    Invests in R&D20

    Network of Partners20

    Pioneers Innovative Features19

    Recognized Brand18

    Comprehensive Training & Cert.17

    Visionary16

    Socially Responsible15

    Likelihood to recommend

    ReliableRobust SecurityResponds QuicklyWarranty Can Count OnConsistently DeliversExpert MaintenanceHigh PerformanceHighly CompatibleDisaster RecoveryManageableEasy to UseUnderstands Your Network Environ.Easily UpgradeableLow Maintenance CostScalableFinancially SoundLow Priced ProductsSets Critical StandardsAccommodating PersonnelUnderstands BusinessWell-RespectedInvests in R&DNetwork of PartnersPioneers Innovative FeaturesRecognized BrandComprehensive Train. & Cert.VisionarySocially ResponsibleLikelihood to recommend

    Importance66595958565248474645404039373229272625232220201918171615

    Estimated Share of Future Networking Budget to Cisco

    France21.7

    Germany20.5

    NA19.2

    UK18.9

    Italy18.9

    India16.8

    Japan9.8

    China7.2

    Sheet3

    FranceGermanyUS/CanUKItalyIndiaJapanIndiaChina

    Cisco Systems21.70%20.50%19.20%18.90%18.90%16.80%9.80%16.80%7.20%

    Microsoft11.10%11.70%16.20%16.40%18.30%13.40%22.90%13.40%15.70%

    HP12.90%13.30%10.00%13.60%14.80%13.00%6.70%13.00%12.60%

    IBM8.80%7.80%8.20%7.70%14.30%13.50%13.80%13.50%17.70%

    Dell7.90%4.30%10.20%8.10%3.70%6.20%8.80%6.20%11.10%

    3Com6.60%6.40%4.40%4.90%5.90%3.60%1.90%3.60%3.40%

    All Others30.90%35.80%31.80%30.50%24.10%33.60%36.20%33.60%32.30%

    TotalTDMBDMCommercialEnterprise

    Cisco Systems17.00%19.00%13.40%13.00%21.20%

    Microsoft15.60%14.30%18.10%17.90%13.30%

    HP12.10%11.60%12.90%12.40%11.70%

    IBM11.00%9.90%12.90%10.70%11.20%

    Dell7.60%7.60%7.60%8.20%7.00%

    3Com4.70%5.20%3.70%5.00%4.30%

    All Others32.00%32.30%31.50%32.70%31.20%

    SORTEDSORTED

    NA/UK

    FRNCE/CHINA/

    GERMNYJAPANINDIA

    ------------------

    32466011203NA/UK/France/GermanyChina/IndiaJapan

    Scalable31.444.730.4175.448.7Reliable60.1

    Easily Upgradeable3739.241.3268.241.4Warranty Can Count On38.1

    Manageable46.550.539.336643.4Robust Security61.8

    Low Maintenance Cost36.652.633.1465.845.1Responds Quickly57.1

    Expert Maintenance58.246.838.9558.238.9Expert Maintenance46.8

    Visionary9.811.526.1657.449.7Consistently Delivers62.1

    Accommodating Personnel20.922.731.1752.542.7Highly Compatible34.4

    Network of Partners14.620.732.2850.939.8High Performance59.8

    Low Priced Products21.45131.7946.539.3Manageable50.5

    Reliable75.460.148.71045.138.5Disaster Recovery57.5

    Recognized Brand9.813.340.9114331Understands Your Network Environ.36.2

    Easy to Use37.951.338.71237.938.7Easy to Use51.3

    High Performance50.959.839.8133741.3Easily Upgradeable39.2

    Disaster Recovery45.157.538.51436.633.1Low Maintenance Cost52.6

    Robust Security6661.843.41531.430.4Scalable44.7

    Comprehensive Training & Cert.14.29.423.71630.830.6Financially Sound23.6

    Highly Compatible52.534.442.71725.927Sets Critical Standards24.9

    Understands Business20.423.326.91821.431.7Low Priced Products51

    Financially Sound30.823.630.61920.931.1Accommodating Personnel22.7

    Well-Respected18.511.139.62920.426.9Understands Business23.3

    Socially Responsible8.315.931.42118.539.6Well-Respected11.1

    Consistently Delivers57.462.149.72217.828.4Invests in R&D10.6

    Responds Quickly65.857.145.12315.728.6Pioneers Innovative Features9.5

    Warranty Can Count On68.238.141.42414.632.2Network of Partners20.7

    Sets Critical Standards25.924.9272514.223.7Comprehensive Training & Cert.9.4

    Invests in R&D17.810.628.4269.826.1Visionary11.5

    Understands Your Network Environ.4336.231279.840.9Recognized Brand13.3

    Pioneers Innovative Features15.79.528.6288.331.4Socially Responsible15.9

  • Client InsightAttribute Importance to Brand Choosers

    Cisco Choosers Competitive Choosers151713811216181213201419 Many Ease Issues Many Expertise Issues16

  • Client InsightStated & Derived Importance

  • The Pricing ConundrumWhy Its a Frustrating AttributeInteresting commentary from the past:Cant you tell me something other than price?Dont worry about price. Its not a driver!

    Its a tough attribute to capture:We treat it like just another attribute when in reality its comparative to the entire set of benefit attributes.Even when treating it like just another attribute, its tough to properly communicate.

    Maybe a more robust approach will make it easier for respondents to answer and more powerful for us to analyze for our clients?

  • The Pricing ConundrumA Different Approach First we look to understand the purchasing philosophy of an organization:Then we have respondents classify brands by how they fit.Traditionally we have a two dimensional analysis as shown earlier brands by attribute.

    This process recasts price from negative benefit to comparative variable - and allows for triangulation the confluence of benefits, price, and brand.

    Prices You Normally PayProducts You Usually SeekPremiumAverageBargainPremium987Average654Bargain321

  • The Pricing DilemmaThe Toughest Assignment of All Pricing research is always dicey.Market dynamics make todays wisdom, tomorrows folly.We design a game then get gamed.Prediction trumps predilection.Its behavior were really after.

  • The Pricing DilemmaApplying Controlled Experiments Heres what a controlled experiment typically looks like:

    The four panels are randomly equivalent and the ONLY difference between them is the price of your product. The respondents task is to review the alternatives and then allocate future purchases across the set of brands.

    PRICESPanel APanel BPanel CPanel DComp 169696969Comp 275757575Comp 359595959Comp 448484848Comp 562626262Comp 669696969Your Brand65707580

  • The Pricing DilemmaApplying Controlled Experiments Results are simultaneously simple and profound. Demand curve based upon price. With financial information and strategic assumptions provides a sound platform for price establishment.But still no slam dunk15171381121618121320141916Chooser SharePrice Points

  • oncept Selection:Are we sayingthe right things?opy Development:Are we sayingthings right? omponent Testing: Do the executions work individually? ampaign Effectiveness:Do the executions work collectively?If we recognize the creative process to be iterative rather than sequential, then the right place to enter the conversation is wherever the conversation happens to be right now.Ad Testing What Does That Mean?In the marketing world, communications is the most costly and important component that routinely escapes rigorous testing.

  • Controlled Experiments Feasibility via LayeringOne Dimensional

    TV A

    600

    22.2

    TV B

    600

    29.8

    TV C

    600

    27.6

    TV D

    600

    21.0

    TV E

    600

    26.3Two Dimensional

    TV A

    TV B

    TV C

    TV D

    TV E

    Print A

    Print B

    Print C200200200

    22.2200200200

    29.8200200200

    27.6200200200

    21.0200200200

    26.327.425.423.0Creative design makes surveys easier for respondents, not harder.

  • Conclusion

  • Respecting Respondents; Enlightening Clients Excellent research incorporates the needs of both clients and respondents.Give respondents boring surveys and they will reward you with boring results.Exhaustion and boredom are the enemies of insight. Treat respondents with the respect and courtesy they deserve. Insight, like beauty, is in the eye of the beholder, and the beholder is your client. Not new, not needed, not insight! Insight is hand crafted. Precisely, procreated. Information close to a decision illuminates which greatly enriches the potential for insight. True insight is a gamble; illumination, our table stakes promise. Your ultimate contribution is as a decision coach. This means knowing your client, knowing your clients business, and knowing your clients critical decisions. Penetrating insight is an uncommon outcome of blunt inquiry.If you arent fond of searching through haystacks, practice designing needles. Expect an insight to often be controversial or contentious. If its nothing more than confirmation of conventional wisdom, its not much of an insight.The best surveys capture what customers have to saynot just what clients want or expect to hear. Further, the former is more rewarding to both parties.

  • Cincinnatis Flatiron Building is the headquarters of Directions Research, Inc. Constructed circa 1900, two years prior to the Flatiron Building in New York, this building originally served as the local home for PPG Industries.

    Completely gutted by a fire in August 1903, the building served primarily as a warehouse until a major renovation and rededication in 1987.An uncommon setting An uncommon contributionDirections Research Inc. 401 East Court Street, Suite 200 Cincinnati, Ohio 45202 513 651-2990 (main) 513 719-2192 (fax) www.directionsresearch.com

    The speaker will be Dennis Murphy. He is 34-year veteran of client-side market research commencing at the now extinct Olympia Brewing Company and culminating as Worldwide Vice President of Market Intelligence at IBM where he built and managed this 1,000 member organization. Dennis failed retirement in early 2005 and joined Directions Research.

    ****

    *Talk to the outline then

    Ground Rules Be prepared for anything Insight on Creating Insight but ask about anythingHiring Practices do we have any thoughts on landing the right folks?Career path how do you find one the right one?Whatever else seems relevant Ask any time

    Follow UpWell be around all afternoonOur contact information is on the handout

    *

    *

    *Clients arent asking for more, theyre asking for better.*Today our client is you. Now just a moment to editorialize. Ive been asking myself, how receptive would I have been in my IBM incarnation to some outsiders offering to teach me about insight. My guess is that my attendance would have been marked by doubt and outright skepticism at the speakers audacity.

    But were not here to teach you anything so much as to reorganize then remind you of what you already at some level know. Insight, which is really enlightenment, is that aha moment when you say to yourself oh ya, thats right, or maybe Id forgotten that. and if something is entirely new well thats a bonus.

    We hope that there is something here for all of you.For those who take away a lot, great.For those who feel short-changed, we really would like your feedback. **Ill use a familiar convention tracking to illustrated designing in insight. Tracking studies have been both a boon and a bane over my 34 year client side career. Ive initiated them and Ive sunset-ed them. The one thing however that I always found most challenging was modifying them. These points illustrate the normal decay process. To say that Ive learned from my mistakes is understatement.

    Hence Relevant Space - an alternative approach.It reintroduces stated importance.It obtains ratings on only familiar brands and important attributes.

    **We ask what brands they know and pick out the five most familiar and then we ask which attributes matter the most. So now weve simplified the task to familiar brands and important attributes. What we gain is respondent interest and attention for the elements rated. What we forego are all of the ratings that they do in brain-dead fashion. Weve always suspected that no more than 50 cells of this 300 cell matrix receive much thought. Now we just go after those. ***Management always asks this question. My answer is did we test what aspect of the ads as illustrated in this model. A typical answer is ya we did a couple of disaster check groups but no may be an even more common response. *Management always asks this question. My answer is did we test what aspect of the ads as illustrated in this model. A typical answer is ya we did a couple of disaster check groups but no may be an even more common response. *Management always asks this question. My answer is did we test what aspect of the ads as illustrated in this model. A typical answer is ya we did a couple of disaster check groups but no may be an even more common response. *Management always asks this question. My answer is did we test what aspect of the ads as illustrated in this model. A typical answer is ya we did a couple of disaster check groups but no may be an even more common response. *Management always asks this question. My answer is did we test what aspect of the ads as illustrated in this model. A typical answer is ya we did a couple of disaster check groups but no may be an even more common response. *****The speaker will be Dennis Murphy. He is 34-year veteran of client-side market research commencing at the now extinct Olympia Brewing Company and culminating as Worldwide Vice President of Market Intelligence at IBM where he built and managed this 1,000 member organization. Dennis failed retirement in early 2005 and joined Directions Research.

    *

Recommended

View more >