Machine Learning Feature Selection question

Question

I was quite wondering, there are a lot of feature selection methods, Like the random forest, stepwise selection, Lasso. But each method gives different important features (some overlapping, but always have some different feature).

For example, now I have x1, x2,x3,x4,x5.

Random Forest choose: x1,x2,x4

LASSO: x1, x4, x5

Stepwise: x1, x4,x3

Could I say that x1 and x4 are definitely important than other variables? since they are chosen by three methods and I wonder besides trying all combinations of the feature selection result, is there a more intuitive way to know which variables are important? or which selection method is the best?

Welcome to Cross Validated! Interesting question and a +1, but are you sure you need to do feature selection? — Dave, Jul 10 '21 at 22:13
Hi, Dave. : ) Yes? to remove the irrelevant variables and have less variance in prediction? — hskdhs, Jul 11 '21 at 01:28
Are you specifying any interaction terms in the LASSO/Stepwise models? — Mark White, Jul 11 '21 at 17:34
Unless you have a really large sample size, it is really hard, often impossible, to deduce the truely influential variables from the data. Even if one method chooses x1, x4 and x5, slight chances in the data could lead to different choices with the same method . How many candidates are there? How many observations? — Bernhard, Aug 20 '22 at 15:14
The trouble [with removing features] is that, while you decrease the variance, you can introduce bias, perhaps enough that you are worse off, despite the decreased variance. — Dave, Aug 20 '22 at 15:15

Machine Learning Feature Selection question

0 Answers0