HYPERPARAMETERS as opposed to model parameters can’t be learn from the data, they are set before training PHASE.

Learning rate:

It determines how fast we want to update the weights during optimization, if learning rate is too small, gradient descent can be slow to find the minimum and if it’s too large gradient descent may not converge(it can OVERSHOOT the minima). It’s CONSIDERED to be the most important hyperparameter.

NUMBER of epochs:

Epoch is defined as one forward pass and one backward pass of all training data.

Batch size:

The number of training examples in one forward/backward pass.

Hyperparameters as opposed to model parameters can’t be learn from the data, they are set before training phase.

Learning rate:

It determines how fast we want to update the weights during optimization, if learning rate is too small, gradient descent can be slow to find the minimum and if it’s too large gradient descent may not converge(it can overshoot the minima). It’s considered to be the most important hyperparameter.

Number of epochs:

Epoch is defined as one forward pass and one backward pass of all training data.

Batch size:

The number of training examples in one forward/backward pass.