In the previous section, you briefly learnt about the notion of ‘cell state’ or an explicit memory of an LSTM. In this lecture, take a look at the next important characteristic of LSTMs.
The gating mechanisms allow modifying the state in certain ways. You’ll learn about how exactly the gating mechanisms work in the next section, though the main idea is that gating mechanisms regulate the information that the network stores (and passes on to the next layer) or forgets.
Now, let’s look at the third and final characteristic of an LSTM cell which helps get rid of the vanishing gradient problem.
The structure of an LSTM cell allows an LSTM network to have a smooth and uninterrupted flow of gradients while backpropagating. This flow is also called the constant error carousel. This third characteristic more or less a result of the first two characteristics and is the reason LSTMs are able to solve the problem of vanishing and exploding gradients.
To summarise, the LSTM is characterised by the following three main properties:
- The cells have an explicit ‘memory’
- The gating mechanisms
- Constant error carousel
In the next section, you’ll study the structure of an LSTM cell to better understand its three properties.
Report an error