9

I'm using the function randomForest in R's randomForest package to do a regression. However, when I'm trying to include an interaction term in the following codes:

library(MASS)
library(randomForest)
Boston_f <- within(Boston, factor(rad))
mdl <- randomForest(lstat ~ rad * . , data = Boston_f)

The result mdl$term does include interaction, but if I peek into the trees that mdl is using, 

getTree(mdl, 1, T)

I cannot find any split variable using interaction term.

Does anyone know how to include interaction term using randomForest or other function?

gung - Reinstate Monica
  • 145,122
Yifei Liu
  • 91
  • 1
    Although this question is asking about R code, I believe it is motivated by a statistical / ML misunderstanding. When that is addressed, the R code specific aspects will be rendered moot. As such, I think this should stay open. – gung - Reinstate Monica Mar 15 '16 at 20:14
  • Questions solely about how software works are off-topic here, but you may have a real statistical question buried here. You may want to edit your question to clarify the underlying statistical issue. You may find that when you understand the statistical concepts involved, the software-specific elements are self-evident or at least easy to get from the documentation. – gung - Reinstate Monica Mar 15 '16 at 20:15

1 Answers1

15

Tree-based models consider variables sequentially, which makes them handy for considering interactions without specifying them. Interactions that are useful for prediction will be easily picked up with a large enough forest, so there's no real need to include an explicit interaction term.

If you believe that the interaction is important, you could manually create the interaction term (for example, defining your formula within the model.frame function, which will create new columns for your interaction terms). Yet in your case this would nearly double the number of variables, as you're creating interactions between rad and every other feature, so it's probably ill-advised.

See also Including Interaction Terms in Random Forest which demonstrates Random Forests' inherent ability to detect interacting variables compared to linear methods.

Community
  • 1
Michael Veale
  • 251