IKH

Optimisation Methods

In the previous section, we learned that to find optimal thetas; we need to optimise the cost function for which we need an optimisation method.

There are two types of Optimisation methods,

  • Closed form solution.
  • Iterative form solution.

Let’s understand this in more detail, in the next lecture.

So, Gradient descent is an iterative form solution of order one. So to compute optimal thetas, we need to apply Gradient Descent to the Cost function, which is given as follows,

∂∂θJ(θ)

In the next lecture, we will compute gradient descent for a simple function and will also be comparing it to the closed form solution. Let’s move to the next lecture.

At 3:59, the prof writes a little outside the screen and he writes θ=θ0.

Gradient descent is an iterative method of optimising an objective function, in our case the cost function, by moving toward the negative of the gradient.

To compute θ1, we saw that the equation will look like this,

Where η is known as the learning rate, which defines the speed at which we want to move towards negative of the gradient. ​

Report an error