When is Ridge regression used and when is Lasso regression (ideally)?

1.	When is Ridge regression used and when is Lasso regression (ideally)?
Answer» It is suggested that in the presence of few variables with MEDIUM / LARGE sized effect, lasso regression can be used. In the presence of many variables with small/medium sized effect, ridge regression can be preferred. Conceptually, lasso regression (L1) does both variable selection and parameter shrinkage, whereas Ridge regression only does parameter shrinkage and end up INCLUDING all the coefficients in the model. In the presence of CORRELATED variables, ridge regression might be the preferred choice. ADDITIONALLY, ridge regression works best in situations where the least square estimates have higher variance. Therefore, it depends on our business goal and model objective as to what is the expectation. Accordingly, decisions can be taken.

Answer»

It is suggested that in the presence of few variables with MEDIUM / LARGE sized effect, lasso regression can be used. In the presence of many variables with small/medium sized effect, ridge regression can be preferred.

Conceptually, lasso regression (L1) does both variable selection and parameter shrinkage, whereas Ridge regression only does parameter shrinkage and end up INCLUDING all the coefficients in the model. In the presence of CORRELATED variables, ridge regression might be the preferred choice. ADDITIONALLY, ridge regression works best in situations where the least square estimates have higher variance.

Therefore, it depends on our business goal and model objective as to what is the expectation.

Accordingly, decisions can be taken.

Discussion

No Comment Found

Related InterviewSolutions

We have below data with 10 transactions. What is the “Lift Ratio” for “if white then blue”?
Transactions#FaceplateColorsPurchased1redwhitegreen2whiteorange3whiteblue4redwhiteorange5redblue6whiteblue7whiteorange8redwhitebluegreen9redwhiteblue10yellow
We have below data with 10 transactions. What is the performance measure “Confidence” for “if white then blue”?
Transaction#FaceplateColorsPurchased1redwhitegreen2whiteorange3whiteblue4redwhiteorange5redblue6whiteblue7whiteorange8redwhitebluegreen9redwhiteblue10yellow
We have below data with 10 transactions. What is the performance measure “Support” for “if white then blue”?
UnivSATTop10AcceptSFRatioExpensesGradRateBrown131089221322,70494CalTech141510025663,57581CMU12606259925,02672Columbia131076241231,51088Comell128083331321,86490Dartmouth134089231032,16295Duke131590301231,58595Georgetown125574241220,12692Harvard140091141139,52597JohnHopkins13057544758,69187MIT138094301034,87091Northwestern126085391128,05289NotreDame125581421315,12294PennState108138541810,18580Priceton13759114830,22095Purdue10052890199,06669Stanford136090201236,45093TexasA&M10754967258,70467UCBerkeley124095401715,14078UChicago129075501338,38087UMichigan118065681615,47085UPenn128580361127,55390UVA122577441413,34992UWisconsin108540691511,85771Yale137595191143,51496
SATAverageSAT score of new freshmenTop10% new freshmen in top 10% of highschool classAccept% of applicants accepted SFRatioStudent to faculty ratioSExpensesEstimated annual expensesGradRateGraduation Rate(%)
Consider universities dataset below. Data for 25 undergraduate programs at business schools in US universities in 1995. The dataset excludes image variables (student satisfaction, employer satisfaction, dean’s opinions, etc.). Given this
We have trained/executed our model with the given dataset. We have noticed that we have used a regression model and it is suffering from multicollinearity. Is it possible to improvise on our model without losing any information?
When is Ridge regression used and when is Lasso regression (ideally)?