Evaluating Models for Customer Churn Prediction

Running head: EVALUATION MODELS 1
Evaluating Models
<Name>
<University Name>

EVALUATION MODELS 2
Evaluating Models
Directions: Use the information provided below to formulate answers for Questions 3 and 4 in
Par1 1 of the Evaluating Models assignment, as specified in the assignment directions.
Contingency Tables (Part 1 - Question 3)
Churn Model
A False True Total
False 2791
TN
59
FP 2850
True 150
FN
333
TP 483
Total 2941 392 3333
Churn Model
B False True Total
False 2636
TN
214
FP 2850
True 106
FN
377
TP 483
Total 2742 591 3333
Churn Model
C False True Total
False 2136
TN
714
FP 2850
True 58
FN
425
TP 483
Total 2194 1139 3333
Outcome Classificatio
n
Actual
Value
Cost
True Negative Churn = false Churn = false $0
True Positive Churn = true Churn = true $2208
False Negative Churn = false Churn = true $2400
False Positive Churn = true Churn = false $48

EVALUATION MODELS 3
Question 1:
The textbook describes a situation where a false positive is worse than a false negative. Describe
a situation from the medical field (e.g., screen testing for a virus) where a false negative would
be worse than a false positive. Explain why it would be worse.
Suppose you need to bring your dog to a vet for a Lyme disease test, the tests would have four
outcomes.
Disease Present Disease not Present
Positive Test Good results Bad Results FP
Negative Test Worse results FN Good Results
A False Negative test -When the test indicates that Lyme disease is not present while indeed the
dog has the disease it is a worse result because the dog won’t get the treatment and it might end
up spreading or even be severe leading to death.
A False Positive test – For this case, a false positive result would imply that the dog will be
treated from the tick-borne disease which is not present, meaning the owner will spend time and
money for the wrong reasons which are not as bad as compared to the False Negative test results.

EVALUATION MODELS 4
Question 2:
In a typical churn model, in which interceding with a potential churner is relatively cheap but
losing a customer is expensive, which error is false negative or a false positive (where positive =
customer predicted to churn)? Explain.
Outcome Classification Actual Value Cost
True Negative Churn = false Churn = false $0
True Positive Churn = true Churn = true $2208
False Negative Churn = false Churn = true $2400
False Positive Churn = true Churn = false $48
Churn Model
Prediction
0 (Negative) 1 (Positive)
Actual
0
Cost = $0
prediction =0
Actual=0
TN
Cost = $48
prediction =1
Actual=0
FP
1
Cost = $2400
prediction =0
Actual=1
FN
Cost = $2208
prediction =1
Actual=1
TP
For a typical churn model where intervention with a potential churner is cheaper than
losing the customer. From the table above the having a false positive churn implies that there
was a positive prediction of churn where intervention was done costing the company $48 while
the actual outcome is that the client did not churn. A False positive churn is cheaper than a false
negative churn – where there was no prediction of churn and no intervention applied to cost the
company $2400 which is more than the FP costing (Olle, 2014).