In regression, a best fit line is a line that fits the given scatter plot in the best way. Let’s see how you can define the notion of a best fit line.
Let’s reiterate what you have learnt so far:
- You started with a scatter plot to check the relationship between sales and the marketing budget.
- You found residuals and the RSS for any given line passing through the scatter plot.
- Then you found the equation of the best fit line by minimising the RSS and also found the optimal values of β₀ and β₁.
So, you know that the best fit line is obtained by minimising a quantity called the residual sum of squares (RSS). You will now be introduced to the concept of a cost function.
Gradient descent
Gradient descent is an optimisation algorithm that optimises the objective function (cost function for linear regression) to reach the optimal solution.
You can learn about cost function and ways to optimise it (minimisation or maximisation) by referring to this link; the topic is covered in detail as part of the optional session. Even though the session is optional, we strongly recommend that you go through it, as gradient descent will also be used for logistic regression and even neural networks.
Let’s now see a demonstration of obtaining a best fit line and finding out the RSS for the marketing spend vs sales example in Excel.
You can download the Excel file from below and look at the analysis carried out by the professor.