In the earlier module on linear regression, you learnt how to build simple models to solve regression problems. Now, before you build upon that learning and understand how to build regression models when there exists a nonlinear relationship between the independent and the target variable, in the forthcoming video, we will quickly review linear regression and learn about a couple of different methods to solve regression problems.
In the above video, Anjali discussed the following equations:
- Simple linear regression equation
Suppose we have ‘n’ observations. Simple linear regression assumes that there is a linear relationship between the input values and the output values. Mathematically, we can express this relationship as:
$$//y_{i\;=\;}\beta_0\;+\;\beta_1\;x_i\;+\;\in_i//$$
where xi is the predictor value (variables that help predict the output), yi is the response value, β0 and β1 are the parameter values or the coefficients, and ϵi is an error or the residual. Here, the parameters are the true parameter values for the data belonging to the population with ‘n’ observations. We want to estimate these parameter values using linear regression.
- Predicted values
$$//\gamma_{i\;=\;}\;b_o\;+\;b_1x_i//$$
In this equation, bo and b1 are the estimated parameter values, which we obtain by fitting a linear regression line, and ^yi is the predicted output value after estimating the model parameters. These estimated values are not exactly equal to the true values, but are very close to them if we get a good fit.
- Residuals
$$//\varepsilon_i\;=\;\gamma_i\;-\;\gamma_i//$$
The error value or residual value for each point is the difference between the actual output (yi) and the predicted output (^yi).
- Residual sum of squares (RSS) or Cost
$$//{\textstyle\sum_{i=1}^n}\;\varepsilon_i^2\;=\;{\textstyle\sum_{i=1}^n}\left(y_{i\;}-y_i\right)^2\;=\;{\textstyle\sum_{i=1}^n}\left(y_i\;-\;b_o\;-\;b_1x_i\right)^2//$$
The sum of the squares of errors is considered the loss/cost function in linear regression, which has to be minimised. The entire linear regression framework is built upon the idea of getting those coefficient estimates that minimise the cost, i.e., RSS. The smaller the value of RSS, the closer is the model fit to the data.
- Root mean squared error
$$//RMSE\;=\;\sqrt{\frac{\sum_{I=1}^n\left(y_i-y_i\right)^2}n}//$$
Root mean square error (RMSE) is the square root of mean squared error. Mean squared error is the variance of the residuals. RMSE tells us how close the actual data points are to predictions made by the model
So, we will now jump straight into some action and build a simple linear regression model in Python. Let’s take a look at it in the next video.
In this video, we have used scikit-learn library of Python for fitting the linear regression line. In the next segment, you will see if there are any methods for getting those coefficient estimates which minimise the cost function.