2

I am doing a multivariate regression analysis with 15 input features, 1 output feature with 1600 samples. I tried SVR, Random forest regressor, KNN, linear, poly and regularized regressions. After trying all, I end up in getting a R² value not more than 0.60.

  1. If I use neural net, is there a possibility to improve the R² value up to 0.90? (cz, I dunno on what logic does ANN work and I read we can do any thing with ANN)

  2. Any suggestions to get a better R² value ?

  3. Is 0.60 is a better R² value? (of course it depends on application and problem type, but I like to know in general)

Ferdi
  • 5,179
Soma Sundaram
  • 31

2 Answers2

3

I know this question is 2 months old and you have probably moved on by now, but for the sake of readers I might suggest that .6 is not a bad R² at all. You say you have 15 features? An R² value of .6 indicates that the correlation of the actual outcome variable values and the predicted variable values is √ .6 = about .77. That's not bad at all.

I'd also like to add that you don't want to try to artificially push R² too high. First of all, the true ability of your predictors to explain the variance in your outcome variable may only be that much. Hence, doing different types of regression over and over until you get something higher might give you a false positive. Related to this is the problem of over-fitting. Sometimes the model finds an idiosyncrasy in the data that allows for really good predictions, but won't generalize to out-of-sample methods. I suggest doing a split of your data set into a train and test set, or even seeing if you can obtain new observations after building on the new model. Check the R² for the test set (correlation of the predicted test set observations and the true observation values, then square it) and you can get a better idea of how predictive your variables are for the outcome.

BKV
  • 410
  • I don't think there is such a thing as a "good" $R^2$ or a "bad" $R^2$. An acceptable value of this statistic is completely dependent on the data and the problem being solved. There's no absolute meaning for these statistics, they are comparative in nature. – Matthew Drury Oct 19 '17 at 01:21
  • 1
    I wouldn't disagree with that. It's hard to shake the dichotomous thinking sometimes! – BKV Oct 19 '17 at 17:28
  • Hi, thanks for the comment. I got the idea of "false positive" cz of trying with different models, to improve R2 value .. – Soma Sundaram Oct 23 '17 at 06:34
1

YES, POSSIBLY

Neural networks have a nice property that they are universal approximators of functions. Loosely speaking, this means that a neural network of sufficient size can approximate a decent function as closely as the designer wants it to. A famous proof of this came from George Cybenko in 1989, and there have been variants of the Cybenko Universal Approximation Theorem since then.

Consequently, a neural network can fit weird shapes that you might not know to tell a linear model to look for by explicitly including nonlinear and/or interaction terms, meaning that the neural network might be able to get a much tighter fit to the data that lowers the square loss and, correspondingly, increases the $R^2$, perhaps to $0.9$ or higher.

What the universal approximation theorems do not comment on is how neural networks perform when they are fit to data, and overfitting is a real concern. However, if you have a bunch of data and an expectation of a very complex relationship between the features and the outcome, a neural network might be a reasonable model.

Without context, it is difficult to evaluate how strong any measure of performance is, so it would not make sense to evaluate your value of $0.6$ except to say that your model is doing better in terms of square loss than a model would do by always predicting the overall mean.

Dave
  • 62,186