1.

So, you have done some projects in machine learning and data science and we see you are a bit experienced in the field. Let’s say your laptop’s RAM is only 4GB and you want to train your model on 10GB data set.

Answer»

What will you do? Have you experienced such an issue before?

In such types of questions, we first need to ask what ML model we have to train. After that, it depends on whether we have to train a model based on Neural Networks or SVM.

The steps for Neural Networks are given below:

  • The Numpy array can be USED to load the ENTIRE data. It will never store the entire data, rather just create a mapping of the data.
  • Now, in order to get some desired data, pass the index into the NumPy Array.
  • This data can be used to pass as an INPUT to the neural network maintaining a small batch size.

The steps for SVM are given below:

  • For SVM, small data SETS can be obtained. This can be done by dividing the big data set.
  • The SUBSET of the data set can be obtained as an input if using the partial fit function.
  • Repeat the step of using the partial fit method for other subsets as well.

Now, you may describe the situation if you have faced such an issue in your projects or working in machine learning/ data science.



Discussion

No Comment Found