We have got a dataset where a number of variables is greater than the

1.	We have got a dataset where a number of variables is greater than the number of observations or rows. Can we use classical Regression techniques here? How would you deal with this situation?
Answer» No, CLASSICAL regression techniques can not be used here. Since a number of variables are greater than a number of observations, it is a high dimension dataset and ordinary least SQUARES cannot be considered for an ESTIMATE as standard DEVIATION and variance will be infinite. We will have to use regression techniques such as Lasso, Ridge, ETC. which will penalize coefficients and will reduce variance and standard deviation. Subset regression and/or stepwise regression can also be explored with a forward step approach.

We have got a dataset where a number of variables is greater than the number of observations or rows. Can we use classical Regression techniques here? How would you deal with this situation?

Answer»

No, CLASSICAL regression techniques can not be used here.

Since a number of variables are greater than a number of observations, it is a high dimension dataset and ordinary least SQUARES cannot be considered for an ESTIMATE as standard DEVIATION and variance will be infinite.

We will have to use regression techniques such as Lasso, Ridge, ETC. which will penalize coefficients and will reduce variance and standard deviation. Subset regression and/or stepwise regression can also be explored with a forward step approach.

Discussion

No Comment Found

Related InterviewSolutions

We have below data with 10 transactions. What is the “Lift Ratio” for “if white then blue”?
Transactions#FaceplateColorsPurchased1redwhitegreen2whiteorange3whiteblue4redwhiteorange5redblue6whiteblue7whiteorange8redwhitebluegreen9redwhiteblue10yellow
We have below data with 10 transactions. What is the performance measure “Confidence” for “if white then blue”?
Transaction#FaceplateColorsPurchased1redwhitegreen2whiteorange3whiteblue4redwhiteorange5redblue6whiteblue7whiteorange8redwhitebluegreen9redwhiteblue10yellow
We have below data with 10 transactions. What is the performance measure “Support” for “if white then blue”?
UnivSATTop10AcceptSFRatioExpensesGradRateBrown131089221322,70494CalTech141510025663,57581CMU12606259925,02672Columbia131076241231,51088Comell128083331321,86490Dartmouth134089231032,16295Duke131590301231,58595Georgetown125574241220,12692Harvard140091141139,52597JohnHopkins13057544758,69187MIT138094301034,87091Northwestern126085391128,05289NotreDame125581421315,12294PennState108138541810,18580Priceton13759114830,22095Purdue10052890199,06669Stanford136090201236,45093TexasA&M10754967258,70467UCBerkeley124095401715,14078UChicago129075501338,38087UMichigan118065681615,47085UPenn128580361127,55390UVA122577441413,34992UWisconsin108540691511,85771Yale137595191143,51496
SATAverageSAT score of new freshmenTop10% new freshmen in top 10% of highschool classAccept% of applicants accepted SFRatioStudent to faculty ratioSExpensesEstimated annual expensesGradRateGraduation Rate(%)
Consider universities dataset below. Data for 25 undergraduate programs at business schools in US universities in 1995. The dataset excludes image variables (student satisfaction, employer satisfaction, dean’s opinions, etc.). Given this
We have trained/executed our model with the given dataset. We have noticed that we have used a regression model and it is suffering from multicollinearity. Is it possible to improvise on our model without losing any information?
When is Ridge regression used and when is Lasso regression (ideally)?