InterviewSolution
Saved Bookmarks
InterviewSolution
| 1. |
Explain learning rate in the context of neural network models. What happens if the learning rate is too high or too low? |
|
Answer» Learning rate is a number that ranges from 0 to 1. It is one of the most important tunable hyperparameters in neural network training models. The learning rate determines how quickly or slowly a neural network model adapts to a given situation and learns. A higher learning rate value indicates that the model only needs a few training epochs and produces rapid changes, whereas a lower learning rate indicates that the model MAY take a long time to converge or may never converge and BECOME stuck on a poor solution. As a result, it is recommended that a GOOD learning rate value be established by TRIAL and error rather than using a learning rate that is too low or too high. In the above IMAGE, we can clearly see that a big learning rate leads us to move away from the desired output. However, having a small learning rate leads us to the desired output eventually. |
|