0

I have a model with 5 basic predictors and all interactions between the predictors themselves. Something like (I'm simplifying here, in reality I have many more variables):

lm(Y ~ poly(X1, 3)*X2*X3*X4*X5)

I want to determine which variables and/or interactions have the largest impact on the model $R^2$ and one way to do this is to extract the principal components.

However, I don't think PCA can tell me if there are some linear combinations of predictors and interactions that significantly contribute to the model $R^2$, especially in presence of polynomial terms and interactions, where the basic predictors can not be interpreted in isolation.

For example, poly(X1, 3) + X2*X3 could have a large explanatory power, but not poly(X1, 3) and X2*X3 in isolation, or X1^2 by itself. See this thread for some background.

How can I identify the predictors, interactions and combinations of terms with the largest explanatory power?

Community
  • 1
Robert Kubrick
  • 4,528
  • "I want to determine which variables and/or interactions have the largest impact on the model R2 one way to do this is to extract the principal components" -- I am not familiar with this use of PCA. How can principal components be used for this purpose? Principal components of what? If you mean principal components of the predictor matrix $X$, then I doubt it can provide any information on the R-squared of its relation to $Y$ (which does not enter PCA). – amoeba Jan 20 '15 at 21:00
  • @amoeba PCR: http://en.wikipedia.org/wiki/Principal_component_regression – Robert Kubrick Jan 20 '15 at 21:42
  • I know what PCR is. I don't think that it can in any way tell you "which variables ... have the largest impact on the model R2". You might be mistaken about what PCR does. – amoeba Jan 20 '15 at 21:44
  • @amoeba ok, I'm not quibbling about PCA or how to use it for this purpose, quite the opposite. How do I identify the most important predictors or combination thereof, when regressing against polynomials and interactions? – Robert Kubrick Jan 20 '15 at 21:47
  • Robert, this I don't know, I am not a specialist in model selection and hope that somebody else is going to provide an answer to your question. My point was only that PCA has nothing to do with your question (as far as I can see), so I would remove the pca tag. – amoeba Jan 20 '15 at 22:18

0 Answers0