岭回归与套索回归
Ridge regression vs Lasso Regression
Lasso 回归或 Elastic-net 回归总是比岭回归好吗？
我已经对几个数据集进行了这些回归，并且我总是得到相同的结果，均方误差在套索回归中是最小的。这仅仅是巧合还是在任何情况下都是如此？
关于这个话题，James、Witten、Hastie 和 Tibshirani 在他们的《统计学习简介》一书中写道：
These two examples illustrate that neither ridge regression nor the
lasso will universally dominate the other. In general, one might expect
the lasso to perform better in a setting where a relatively small
number of predictorshave substantial coefficients, and the remaining
predictors have coefficients that are very small or that equal zero.
Ridge regression will perform better when the response is a function of
many predictors, all with coefficients of roughly equal size. However,
the number of predictors that is related to the response is never
known apriori for real data sets. A technique such as cross-validation
can be used in order to determine which approach is betteron a
particular data set. (chapter 6.2)
每个问题都不一样。在套索回归中，算法试图删除没有任何用处的额外特征，这听起来更好，因为我们也可以用更少的数据很好地训练，但处理有点困难，但在岭回归中，算法正在尝试使这些额外功能的效率降低但不完全删除它们更容易处理。
Lasso 回归或 Elastic-net 回归总是比岭回归好吗？
我已经对几个数据集进行了这些回归，并且我总是得到相同的结果，均方误差在套索回归中是最小的。这仅仅是巧合还是在任何情况下都是如此？
关于这个话题，James、Witten、Hastie 和 Tibshirani 在他们的《统计学习简介》一书中写道：
These two examples illustrate that neither ridge regression nor the lasso will universally dominate the other. In general, one might expect the lasso to perform better in a setting where a relatively small number of predictorshave substantial coefficients, and the remaining predictors have coefficients that are very small or that equal zero. Ridge regression will perform better when the response is a function of many predictors, all with coefficients of roughly equal size. However, the number of predictors that is related to the response is never known apriori for real data sets. A technique such as cross-validation can be used in order to determine which approach is betteron a particular data set. (chapter 6.2)
每个问题都不一样。在套索回归中，算法试图删除没有任何用处的额外特征，这听起来更好，因为我们也可以用更少的数据很好地训练，但处理有点困难，但在岭回归中，算法正在尝试使这些额外功能的效率降低但不完全删除它们更容易处理。