1.

What are Exploding Gradients and Vanishing Gradients?

Answer»
  • EXPLODING GRADIENTS: Let us say that you are training an RNN. Say, you saw exponentially growing error gradients that accumulate, and as a result of this, very large updates are made to the neural network model weights. These exponentially growing error gradients that update the neural network weights to a great extent are called Exploding Gradients.
  • Vanishing Gradients: Let us say again, that you are training an RNN. Say, the slope became too small. This problem of the slope becoming too small is called Vanishing Gradient. It causes a major increase in the training time and causes POOR performance and extremely low accuracy.


Discussion

No Comment Found

Related InterviewSolutions