You have come a long way! Let’s look at a summary of all that you have covered in the last two sessions.
In the first session, we went through the concepts of boosting and studied one of the earliest boosting algorithms, AdaBoost. We went through the process of updating the distribution of the data points by changing the probabilities attached to them and deciding the weights attached to the individual models that fit on this distribution.
In the next session, we went through Gradient Boosting and a modification of it, XGBoost. We were introduced to the intuition of gradient descent in the regression setting when we used the square loss function to close in on the residues. We developed from here on the Gradient Boosting algorithm in which each subsequent model trains on the gradients of the loss function. XGBoost follows the same procedure on trees with additional features like regularization and parallel tree learning algorithm for finding the best split.
You can download the lecture notes for this module from the link below: