In the previous segment, you built the CTR prediction model. However, there was no discussion on how the model is performing and the various evaluation metrics to test the model. So, in the upcoming video, Ajay will take you through the model evaluation part.
First, the coefficients and the intercept of the model can be obtained using the following code:
As explained by Ajay, the data set used here is very unbalanced as most users would not click on the Ad. So, if any model is predicting 0 for all the values, the accuracy would be still 85% as most of the ‘clicks’ are values 0. So, in such cases, the ROC curve and precision recall curve can be used to test the correctness of the model. The ROC curve for the present case is as follows:
Another important metric is the Area Under Curve (AUG), which is defined as the area under the ROC curve. As explained by Sajan, AUC should be greater than 0.5 for a good model. The AUC obtained for the model was 0.61, which indicates that the model is performing good.
The precision-recall (PR) curve is also used as a metric to ascertain the correctness of the model. The PR curve is as follows:
An important point to note here is that there is always a trade-off between precision and recall. When one increases, the other decreases. Now that you have learnt about the model parameters, you also need to test your model on the test data. For this, you can use the following commands:
The accuracy turns out to be 82.39%, which indicates that the model is working well on the test data. Also, the AUC for the model on the test data is 0.60, which is very close to the value obtained for the training data; this further indicates that the model is predicting well.
Report an error