In the previous segment, you learnt about two simple forecasting models and also you learnt to build these models on an air passenger traffic dataset using Python. In this segment, you will learn about some popular error measures which will help you to evaluate the accuracy of the models.
Note:
In this video at (1:48) the SME writes ’10-8′ as ‘-2’ which is incorrect. It should be ‘2’.
Let’s quickly summarise the three popular error measures that you learnt in the video above.
- Mean Forecast Error (MFE): In this naive method, you simply subtract the actual values of the dependent variable, i.e., ‘y’ with the forecasted values of ‘y’. This can be represented using the equation below.
$$\\\;MFE=\frac1n{\textstyle\sum_{i=1}^n}\begin{pmatrix}yactual-\widehat y&forecast\end{pmatrix}\\$$
- Mean Absolute Error (MAE): Since MFE might cancel out a lot of overestimated and underestimated forecasts, hence measuring the mean absolute error or MAE makes more sense as in this method, you take the absolute values of the difference between the actual and forecasted values.
$$\\MAE=\frac1n{\textstyle\sum_{i=1}^n\begin{vmatrix}yactual-\widehat y&forecast\end{vmatrix}}\\$$
- Mean Absolute Percentage Error (MAPE): The problem with MAE is that even if you get an error value, you have nothing to compare it against. For example, if the MAE that you get is 1.5, you cannot tell just on the basis of this number whether you have made a good forecast or not. If the actual values are in single digits, this error of 1.5 is obviously high but if the actual values are, say in the order of thousands, an error of 1.5 indicates a good forecast. So in order to capture how the forecast is doing based upon the actual values, you evaluate mean absolute error where you take the mean absolute error (MAE) as the percentage of the actual values of ‘y’.
$$\\MAPE=\frac{100}n{\textstyle\sum_{i=1}^n}\left|\frac{y_{i-{\widehat y}_i}}{y_i}\right|\\$$
You learnt about two other popular error measures. These were –
- Mean Squared Error (MSE): The idea behind mean squared error is the same as mean absolute error, i.e., you want to capture the absolute deviations so that the negative and positive deviations do not cancel each other out. In order to achieve this, you simply square the error values, sum them up and take their average. This is known as mean squared error or MSE which can be represented using the equation below.
- Root Mean Squared Error (RMSE): Since the error term you get from MSE is not in the same dimension as the target variable ‘y’ (it is squared), you deploy a metric known as RMSE wherein you take the square root of the MSE value obtained.
Now that you have a good understanding of the error measures, in the next video you will learn how to calculate the error terms, primarily MAPE and RMSE for the naive model that you had built on the air passenger traffic dataset in the previous segment.
Next, let’s also look at the RMSE and MAPE values for the forecast built using the simple average method.
So far you learnt the two basic forecasting techniques which are –
- Naive method
- Simple average method
In the simple average method, we take the average of the complete time-series data. What if we take only the last few set of observations to predict the future, as the last observation has more impact on the future than the first observation. This is where Simple moving average comes into the picture. You will learn more about this in the next segment.
You learnt the theoretical concepts of two forecasting techniques, Naive method and Simple average method and built the model in python. Now, it is time for you to build these models on a real-life dataset using Python.