Normalisation of Images
Next, we move on to normalisation. You might remember normalisation from the previous sessions. It is a common technique used with all types of data. Let’s hear Rohit talk about normalisation in this context.
Note: Formula for normalisations are:
(image−np.min(image))/(np.max(image)−np.min(image)) and (image−np.percentile(image,5))/(np.percentile(image,95)−np.percentile(image,5))
Some of the brackets are missing in the video.
Why do we Normalise?
Normalisation makes the training process much smoother. This is an important preprocessing step, so let’s discuss it briefly.
For example, let’s say you have some data points X1, X2, X3,…, Xn. The range of values of most data points is between (say) -10 to 10, but a few data points (say X11 and X18) have values ranging from -900 to 1000.
Now, in backpropagation, the gradients are (directly or indirectly) related to the derivatives f′(x) where f is the activation function. Say you are using a sigmoid activation. In sigmoid, the value of f′(x) at X=-800 and X=900 is almost zero, but it is a small positive number X=-1 and +1.
This makes the gradient with respect to X11 and X18 drop to almost zero, and so the weight updates cannot happen in the right direction. Although sigmoid is rarely used in modern deep learning architectures, this problem arises in other activation functions as well and can be reduced using normalisation.
Outliers
In the case, you have outliers in data and if you normalise by the equation (X – Xmin )/ (Xmax – Xmin ), then your ”normal” data will scale to a very small range, something like O to 1. Therefore, it is suitable to normalise using the percentile as explained, by Rohit.
Try answering these questions about normalisation.
We have now transformed the original data to make it more suitable as input for training later.
Coming Up
In the upcoming page, we will explore another important function of data pre-processing- creating extra data out of existing data. This falls under the umbrella term of data augmentation.
Additional Reading
- Working with Neural Networks always involves using tricks to speed up computation. This excellent paper uses a technique called ‘pixel normalisation’ to modify text data into an image form, in order to enable fast processing.
Report an error