We have a dataset comprising of variables having more than 30% missing

1.	We have a dataset comprising of variables having more than 30% missing values. Let’s say, for example, we have 100 variables and 16 variables have missing values of more than 30%. How will you deal with this scenario?
Answer» ASSIGN a unique category to MISSING values, who KNOWS the missing values might decipher some trend. Perform exploratory analysis to visualize and understand them better. We can remove them blatantly. We can sensibly check their distribution with the target variable, and if FOUND any pattern we’ll keep those missing values and assign them a new category while removing others. In a nutshell, while handling missing values, we will have to understand data first and BASED on that, various mechanisms can be performed to treat them. There is no specific rule for a particular scenario. It is data-driven and context specific.

We have a dataset comprising of variables having more than 30% missing values. Let’s say, for example, we have 100 variables and 16 variables have missing values of more than 30%. How will you deal with this scenario?

Answer»

ASSIGN a unique category to MISSING values, who KNOWS the missing values might decipher some trend. Perform exploratory analysis to visualize and understand them better.
We can remove them blatantly.
We can sensibly check their distribution with the target variable, and if FOUND any pattern we’ll keep those missing values and assign them a new category while removing others.

In a nutshell, while handling missing values, we will have to understand data first and BASED on that, various mechanisms can be performed to treat them.

There is no specific rule for a particular scenario. It is data-driven and context specific.

Discussion

No Comment Found

Related InterviewSolutions

We have below data with 10 transactions. What is the “Lift Ratio” for “if white then blue”?
Transactions#FaceplateColorsPurchased1redwhitegreen2whiteorange3whiteblue4redwhiteorange5redblue6whiteblue7whiteorange8redwhitebluegreen9redwhiteblue10yellow
We have below data with 10 transactions. What is the performance measure “Confidence” for “if white then blue”?
Transaction#FaceplateColorsPurchased1redwhitegreen2whiteorange3whiteblue4redwhiteorange5redblue6whiteblue7whiteorange8redwhitebluegreen9redwhiteblue10yellow
We have below data with 10 transactions. What is the performance measure “Support” for “if white then blue”?
UnivSATTop10AcceptSFRatioExpensesGradRateBrown131089221322,70494CalTech141510025663,57581CMU12606259925,02672Columbia131076241231,51088Comell128083331321,86490Dartmouth134089231032,16295Duke131590301231,58595Georgetown125574241220,12692Harvard140091141139,52597JohnHopkins13057544758,69187MIT138094301034,87091Northwestern126085391128,05289NotreDame125581421315,12294PennState108138541810,18580Priceton13759114830,22095Purdue10052890199,06669Stanford136090201236,45093TexasA&M10754967258,70467UCBerkeley124095401715,14078UChicago129075501338,38087UMichigan118065681615,47085UPenn128580361127,55390UVA122577441413,34992UWisconsin108540691511,85771Yale137595191143,51496
SATAverageSAT score of new freshmenTop10% new freshmen in top 10% of highschool classAccept% of applicants accepted SFRatioStudent to faculty ratioSExpensesEstimated annual expensesGradRateGraduation Rate(%)
Consider universities dataset below. Data for 25 undergraduate programs at business schools in US universities in 1995. The dataset excludes image variables (student satisfaction, employer satisfaction, dean’s opinions, etc.). Given this
We have trained/executed our model with the given dataset. We have noticed that we have used a regression model and it is suffering from multicollinearity. Is it possible to improvise on our model without losing any information?
When is Ridge regression used and when is Lasso regression (ideally)?

We have a dataset comprising of variables having more than 30% missing values. Let’s say, for example, we have 100 variables and 16 variables have missing values of more than 30%. How will you deal with this scenario?

Discussion

No Comment Found

Related InterviewSolutions

Reply to Comment