© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Denis V. Batalov, PhD

AWS Solutions Architect, EMEA

November 30, 2016

Predicting Customer Churnwith Amazon Machine Learning

@dbatalov

MAC307

Customer churn

Machine learning

Science

• Computer Science

• Statistics

• Neuroscience

• Operations Research

Artificial Intelligence

• Rule extraction from data

• Inspired by human learning

• Adaptive algorithms

Engineering

• Training: Data Models

• Prediction: Models Forecast

• Decision: Forecast Actions

ML: Robotics

ML: Robotics

ML: Image recognition

Supervised learning

Supervised learning

Input Outcome

Supervised learning

Input Outcome

Input

Input

Input

Outcome

Outcome

Outcome

Supervised Learning

Input Outcome

Input

Input

Input

Outcome

Outcome

Outcome

Supervised

Learning

known historical data

Amazon ML

Supervised learning

Input Outcome

Input

Input

Input

Outcome

Outcome

Outcome

Supervised

Learning

Unseen Input Same Outcome

known historical data

Amazon ML

Amazon Machine Learning service

Amazon Machine Learning service

Amazon Machine Learning service

Amazon Machine Learning service

Telco churn dataset

• US telco customers, their cell phone plans and usage

• 21 attributes, 3333 rows:

• Customer: State, Area_Code, Phone

• Plan: Intl_Plan, VMail_Plan

• Behavior: VMail_Messages, Day_Mins, Day_Calls,

Day_Charge, Eve_Mins, Eve_Calls, Eve_Charge,

Night_Mins, Night_Calls, Night_Charge, Intl_Mins,

Intl_Calls, Intl_Charge

• Other: Account_Length, CustServ_Calls, Churn

Telco churn dataset

• US telco customers, their cell phone plans and usage

• 21 attributes, 3333 rows:

• Customer: State, Area_Code, Phone

• Plan: Intl_Plan, VMail_Plan

• Behavior: VMail_Messages, Day_Mins, Day_Calls,

Day_Charge, Eve_Mins, Eve_Calls, Eve_Charge,

Night_Mins, Night_Calls, Night_Charge, Intl_Mins,

Intl_Calls, Intl_Charge

• Other: Account_Length, CustServ_Calls, Churn

Telco churn dataset

KS, 128, 415, 382-4657, 0, 1, 25, 265.100000, 110, 45.070000, 197.400000, 99, 16.780000, 244.700000, 91, 11.010000, 10.000000, 3, 2.700000, 1, 0

OH, 107, 415, 371-7191, 0, 1, 26, 161.600000, 123, 27.470000, 195.500000, 103, 16.620000, 254.400000, 103, 11.450000, 13.700000, 3, 3.700000, 1, 0

NJ, 137, 415, 358-1921, 0, 0, 0, 243.400000, 114, 41.380000, 121.200000, 110, 10.300000, 162.600000, 104, 7.320000, 12.200000, 5, 3.290000, 0, 0

OH, 84, 408, 375-9999, 1, 0, 0, 299.400000, 71, 50.900000, 61.900000, 88, 5.260000, 196.900000, 89, 8.860000, 6.600000, 7, 1.780000, 2, 0

OK, 75, 415, 330-6626, 1, 0, 0, 166.700000, 113, 28.340000, 148.300000, 122, 12.610000, 186.900000, 121, 8.410000, 10.100000, 3, 2.730000, 3, 0

AL, 118, 510, 391-8027, 1, 0, 0, 223.400000, 98, 37.980000, 220.600000, 101, 18.750000, 203.900000, 118, 9.180000, 6.300000, 6, 1.700000, 0, 0

Console: Creating datasource for Amazon ML

Console: Creating datasource for Amazon ML

Console: Building the Amazon ML model

Recipe

{ "groups": {

"NUMERIC_VARS_NORM": "group('Intl_Charge','Night_Calls','Day_Calls','Eve_Calls','Eve_Mins','Intl_Mins','VMail_Message','Intl_Calls','Day_Mins','Night_Mins','Day_Charge','Night_Charge','Eve_Charge','Account_Length')” },

"assignments": {},

"outputs": [

"ALL_BINARY",

"State",

"Area_Code",

"normalize(NUMERIC_VARS_NORM)",

"CustServ_Calls"

]

}

Recipe: normalize() function

Account_Length Normalized Value

128 0.808771865

107 -0.047574816

137 1.175777586

84 -0.985478323

75 -1.352484044

118 0.400987732

Building the Amazon ML model

Cost of errors

• Cost of customer churn and acquisition (false negative):

• Foregone cash flow

• Advertising costs

• POS and sign-up admin costs

• Customer retention cost (false + true positive)

• Discounts

• Phone upgrades

• Etc.

Financial outcome of applying a model

Prior Churn Churn Cost Cost without ML

14.49% $500.00 $72.46

Financial outcome of applying a model

Prior Churn Churn Cost Cost without ML

14.49% $500.00 $72.46

False Negative True + False Pos Retention Cost Cost with ML

4.80% 12.10% + 14.30% $100.00 $50.40

Financial outcome of applying a model

Prior Churn Churn Cost Cost without ML

14.49% $500.00 $72.46

False Negative True + False Pos Retention Cost Cost with ML

4.80% 12.10% + 14.30% $100.00 $50.40

• Threshold 0.3 0.17

• $22.06 of savings per customer

• With 100,000 customers over $2MM in savings with ML

What’s next?

• https://aws.amazon.com/getting-started/projects/build-

machine-learning-model/

• https://aws.amazon.com/machine-learning/developer-

resources/

• https://github.com/dbatalov/cost_based_ml

Thank you!

Denis V. Batalov, PhD

AWS Solutions Architect, EMEA

@dbatalov

Remember to complete

your evaluations!