1.

What are different ways to check model performance? Explain one of them briefly?

Answer»

Response: 

There are various ways to check the performance of a model that is being developed. Some of the key approaches are as follows: 

  1. CONFUSION Matrix 
  2. Accuracy 
  3. Precision and Recall 
  4. F1 score 
  5. ROC or Receiver Operating Characteristic Curve 
  6. Precision-Recall Curve vs ROC curve 

For example, we can consider a binary classification scenario and will explain Precision / Recall in that case. 

Assume that, there are 100 positive cases among 10,000 cases. You want to predict which ones are positive, and you pick 200 to have a better chance of catching many of the 100 positive cases. You record the IDs of your predictions, and when you get the actual results you sum up how many times you were correct or incorrect. There are four ways of being correct or incorrect. 

  • TN / True Negative: case was negative and predicted negative 
  • TP / True Positive: case was positive and predicted positive 
  • FN / FALSE Negative: case was positive but predicted negative 
  • FP / False Positive: case was negative but predicted positive 

Predicted Negative
Predicted Positive
Actual Negative Cases
9770 (TN)
130 (FP)
Actual Positive Cases
30 (FN)
70 (TP)

Now in the above example, if we COMPUTE

  • What percent of your predictions were correct? Answer: the "accuracy" was (9770+70) out of 10,000 = 98.4% 
  • What percent of the positive cases did you catch? Answer: the "recall" was 70 out of 100 = 70% 
  • What percent of positive predictions were correct? Answer: the "precision" was 70 out of 200 = 35% 


Discussion

No Comment Found