1

I am cleaning data that I will use with machine learning prediction algorithms.

Several of my variables in my data set are sums of other variables. eg) given variables x1, x2, x3, x3=x2+x1 or even x4= x5+x6+...x10.

I feel like I should remove these variables but am not sure. Is there any reason to keep variables like this or should you always remove these kinds of variables? Related question, how does collinearity affect your machine learning algorithm?

sma
  • 233
  • What do you mean by "your machine learning algorithm"? Do you have a specific one in mind? If not, your related question may be too broad to be answerable. – whuber Feb 16 '18 at 17:54
  • Random Forests, logistic regression, neural networks, svm and maybe a couple of more. – sma Feb 16 '18 at 19:31
  • Yes, that's too broad, because the answer will differ depending on the technique. The first part of your question may be specific enough to support a focused discussion. – whuber Feb 16 '18 at 19:40

1 Answers1

0

In general these derived attributes should be removed since they tell no new information.

sma
  • 233
  • This isn't always the right advice. If theory suggests these derived variables play specific roles, then they should not be removed. – whuber Feb 20 '18 at 04:37