The Fallacy of the Net Promoter Score: Customer Loyalty Predictive Model

Mohamed Zaki, Dalia Kandeil, Andy Neely and Janet McColl-Kennedy

Agenda

CustomerLoyaltyMeasurement

NPSCritiques

ResearchFramework

AnalysisandFindings

Contribution

Customer Loyalty Measurement

2. NetPromoterScore(NPS)

KeyQuestion

Overall,howsatisfiedareyouwiththisbrand/product/service?

Rationale

1.CustomerSatisfaction(CSAT)

“CSATisadaptable,itcanbeappliedwitharangeofscalestomeasurespecificpartsoftheexperience.”

KeyQuestion

Howlikelyareyoutorecommendourcompany/product/servicetoyourfriends&colleagues?

Rationale

NPS=%Promoters- %Detractors“Loyaltyismoreefficient thanacquisition,

thereforefocusonthosewho

promoteyou”

Single-question customermetricshavebecomeverypopularastoolstomeasurecustomerloyalty

Customer Loyalty Measurement

3.CustomerEffortScore(CES)

KeyQuestion

Howmucheffortdidyoupersonallyhavetoputforthtohandleyourrequest?Scoresrangefrom1(loweffort)to5(higheffort)

Rationale

“Justmakeiteasy forme”

Eachclaimstobetheonemetricthatmanagersneedtomeasure,monitor,andacton

AttitudinalLoyalty

BehaviouralLoyalty

Expressedbyrepurchasingfrequency,overallmonetaryexpenditure→Transactionaldata

Definedasreferralintention,word-of-mouthpublicity,opinionfeedback

→Surveydata

Literature Review: Customer LoyaltyConsidered to powerfully impact a firm’s performance and competitive advantage

MeasurementMethods

Definition “Abuyer’soverallattachmentordeepcommitment toaproduct,service,brand,ororganization”(Lametal2004)

PerformanceMeasurement

BigDataMarketing

Customer satisfaction and loyalty are one of central key focus of marketing theory and practice. Loyalty is a complex psychological construct (Pollack and Alexandrov 2013)

The Net-promoter Score Critiques Regarded as an attitudinal measure for assessing customer loyalty

#1Noempiricalevidencethatattitudinal loyaltymetricssignificantlycorrelatewiththerelativechangeinrevenuewithintherespectiveindustry

#2 Singlemetricalonecannotpredictcustomerloyaltyandunlikelytodeliverinsightstomanagersforactions

#4

NPSisnotasufficientapproachtothecustomerloyaltymeasurementandbecauserecommendationsalonearenotenoughtodrivebusinesssuccess

CustomerscouldgivehighNPSscores(promoters),whileafirmcouldloseapercentageoftheircustomers’base.

#3

Thesemetricsmisinformmanagersanddivertthemawayfrommarketingactions#5NPSdoesnotprovidesuchaprescriptionforfirmstodiagnosetheunderlyingcausalfactorsoftheircustomers.#6

(Grisaffe2004;Keiningham etal.2007;KristensenandEskildsenn 2011;MorganandRego 2006;PollackandAlexandrov 2013;Pingitore etal.2007)

Research Framework and Methodology

AdaptedfromCross-Industry-Standard-ProcessforDataMining(Chapmanetal.2000;Villarroel Ordenes etal.2014)

Business Understanding- Finning Case StudyCompanyinterviewswereconductedtograspcurrentloyaltyassessmentprocess

CurrentFeedbackProcess

WeeklyCustomerFeedbackSurvey

ResponsesRecorded

MonthlyManualAnalysis

Issues

Onlyrandom2%ofcustomersaresurveyed

Surveyrespondentisnotastrategicdecisionmaker

CustomerloyaltyisassessedontheNPSasastand-alonemetric

Nolinkagewithcustomer’sactualspendingpatterns

Data Understanding – Data SourcesData from multiple sources were gathered for the purpose of this study

CustomerComments

NPSRating

AttitudinalData

TransactionalData RegionalLocation

BehavioralData DemographicData

• DataTimeframe– limited toabout3years(2012,2013,2014,2015Q1)

• DataSize– over1millionrecords(1,044,512)ontransactionallevel

• TransactionalData– Sales,ProductSupport(PS),CustomerServiceAgreement(CSA)

• Sales– New,UsedorLeasetransactiontypes

• PSandCSA– PartsandServicetransactiontypes

Data Preparation – RFM AnalysisCustomers were scored based on their actual spending behavioural patterns

Technique DataEmployed Purpose

RFMAnalysis PSTransactions Assessing CustomerProfitability

ClusteringusingK-means RFMScores

Groupingcustomerswithsimilar

profitabilitytogether

RFM Analysis – ResultsCustomers were scored based on their actual spending behavioural patterns

K-means Clustering – ResultsCustomers were segmented into 11 groups to mimic the NPS scale

ClusterNumber

AverageRFM

CorrespondingNPSScale

NPSCategory

5 111 0

Detractor

9 112 1

8 221 2

4 222 3

10 223 4

11 332 5

3 333 6

Passive7 334 7

6 343 8

2 344 9Promoter

1 555 10

RFM Clusters Vs. Survey NPSComparison between each NPS category and RFM equivalent was performed

MisclassifiedNPS 2012 2013 2014

Promoter→Detractor 291 522 524

Passive→Detractor 126 185 218

Promoter→Passive 135 147 104

Passive→Promoter 44 18 32

Detractor →Promoter 10 2 3

Detractor →Passive 9 8 10

%ofcustomermisclassified 72% 85% 82%

Data Preparation – Defining Active Customer“Active customer” was generated due to lack of formal definition

AveragenumberofdaysbetweentwoconsecutivetransactionsforallcustomersDefine

Diff_Recency=Dateoflastesttransaction– DateofsecondlatesttransactionCalculate

IfcustomerRecency<Avg.Diff_Recency→LoyalElse →ChurnerGenerate

Ifcustomershavenotboughtwithin71days→ChurnerTherefore,abletopredictwhethercustomersarelikelytochurnResult

46%

54%

Churner

Loyal

Data Preparation – Quantifying Text AnalyticsComplaint status of survey comments were transferred into quantitative scores

OverallSatisfaction AdditionalComments

10

PartServiceisthebestpartofXXXXandthatoneIwillrateveryhigh.Howeverworkshopcostsalotofmoneyandisawasteoftime.Theproblemisthat[company]isonlyinterestedindoingbusinesswithbigcompaniesanddoesn´tcareaboutsmallone

CommentScore=Compliments– Complaints– Suggestions

Calculatecustomeraverageacrossallsurveyentries

ComplaintStatus

Complainer

Neutral

Satisfied

1

2

3 Binscoresinto3satisfaction levelcategories

63%15%

22%

ComplainerNeutralSatisfied

Data Modelling and Prediction Data was split into three sets to test the prediction accuracy of the loyalty model

TrainingData(60%)

Testing Data(30%)

Validation Data(10%)

Insight HighlightsCustomers’ NPS score and purchasing pattern do not match across 3 years

5%

25%

70%

PerceivedNPS

DetractorPassivePromoter

46%

54%

ActualBehavior

ChurnerLoyal

Churners vs. Sales VolumeA root cause analysis of complaining customers was performed

CustomerID Sales Volume(£)

ComplaintStatus

Comment Resource

FIN0408720 1,333,736 Complainer

Generally,it'salongwaywithgettingfeedbackandpartorderanddeliverysometimes

Communication

Droptheprices,Evenfromlastyearthepricesseemtoohigh PriceValue

FIN0053511 949,016 Complainer

Reducethehourlyrateabit.Iwouldlikeittobetowards40poundsperhour.

Price Value

Makeitcheaper PriceValue

FIN0343268 425,586 Complainer

Justkeepusmoreinformed.Notjustleavingusherewaiting,wonderingifsomeone'scomingornot.

Communication

FIN0391511 361,820 Complainer Theyshouldimproveon-sitejobtraining,improvequalitycontrol.

ProcessAdherence

Conclusion• ThestudycontributestowardsprovingtheunreliabilityoftheNPSasasingleloyaltymeasureswithinB2Bcomplexserviceorganizations

• Theframeworkintegratesamultitudeofdemographic,behavioral,andattitudinalcustomerdatawhenassessingcustomerloyalty

• Thecombinationofmultipledatawhenassessingcustomerloyaltysupportscriticismintheliteratureofusingsingleloyaltymetric

• Thepredictionmodelisdevelopedusingdifferent bigdatatechniquestopredictcustomerloyaltyandidentifycustomerswhohave"churned”

• Weextended thelinguistictextminingapproachtodeterminethecomplaintstatusandemotionsanddividescustomersintogroupsofcomplainer,neutralorsatisfied

Forthcoming Webinars

Date14:30hr GMT

Topic Invitedspeaker

December12th FeedbackfromtheFrontline:EngagingFront-LineEmployeesinService Contracts

FlorianUrmetzer

2017

January9th TheFallacyoftheNetPromoterScore:CustomerLoyaltyPredictiveModel

MohamedZaki

February13th ClassificationofNoisyData:AnApproachBasedonGeneticAlgorithmsandVoronoi Tessellation

Abdul Khan

March13th TheEcosystemValueFramework:SupportingManagerstoUnderstandValueExchangebetweenCoreBusinessesinServiceEcosystems

FlorianUrmetzer