In a previous session on model evaluation, you learnt about various model evaluation measures, such as accuracy, sensitivity, specificity, KS statistic, etc. Now, let’s look at some more measures commonly used for model evaluation.
Note
: At 02:29 our SME’s says “%Goods in X-axis and %Bad in Y-axis”, please note that it should be %Bad in X-axis(FPR) and %Good in Y-axis(TPR).
Hindol also talked about some measures other than sensitivity and specificity. If you want to go deep and learn about them, you can access them through the following links:
The following shows the ROC curve for 3 cases, i.e. the random model (orange line), the ideal model (grey line), and the real model(blue line), in our case, the telecom churn model.
Clearly, the perfect model is pretty much a right triangle, whereas the random model is a straight line. Basically, a model that rises steeply is a good model.
Another way of saying that is — it will have a higher area under the curve. So, the Gini coefficient, which is nothing but a fancy word and is given by –
will be high for a good model.