By model optimization, we mean to build/refine the model in such a way to be as realistic as it can be. It should reflect the real-life situation as closely as possible. When we apply a model to the real-world data, it should give the expected results. So optimization is required. This is achieved by capturing some significant or key components from the dataset.

There are some TOOLS available in the market for optimizing the models. One such tool is the ‘TensorFlow Model Optimization Toolkit’. There are THREE major components in model optimization:

1. An objective function.
2. Decision VARIABLES
3. Constraints

An objective function is a function that we need to optimize for model Optimization. The solution to a given optimization problem is nothing but the set of values of the decision variables. These are those values of the decision variables for which our objective function reaches its expected optimal value. The values of the decision variables are restricted by the constraints.

The classification of optimization problems is based on the nature of our objective function and the nature of given constraints. In an unconstrained optimization problem, there are no constraints and our objective function can be of any kind - linear/NONLINEAR. In the linear optimization problem, our objective function is linear in variables and the given constraints are also linear.

In a quadratic optimization problem, our objective function quadratic in variables and the given constraints are linear. In a nonlinear optimization problem, our objective function is an ARBITRARY function that is nonlinear of the given decision variables.

The given constraints can be linear or they can be nonlinear. The objective of model optimization is to find the optimal values of the given decision variables.