2

I am trying to compare models using K-Fold-CV using the regsubsets function in R.

By default, it states that the ideal model is determined by the $RSS$.

I wished to change this parameter such that ideal model selection was performed on the basis of the $R^2$ value. However, I noticed that:

$R^2 = 1 - (RSS/TSS)$, and $TSS = \sum \big(y_i-\bar y\big)^2$. And therefore, is ideal model selection based on $RSS$ essentially equivalent to ideal model selection by $R^2$, given that the $TSS$ value is constant between test data-sets?

Dave
  • 62,186
h3ab74
  • 133

1 Answers1

2

They are equivalent for the exact reason that you mention: one is a monotonic transformation of the other. Just keep in mind that we want to maximize $R^2$, while we want to minimize the residual sum of squares.

Since we typically want to find the best model for a set of data, we typically have a constant $TSS$.

Dave
  • 62,186
  • Note that $R^2$ can be made quite large (close to $1$) by adding many parameters that fit the noise but would not generalize. Residual sum of squares suffers from the analogous issue of being able to be made quite small (close to $0$) by using many parameters that fit to the noise in a way that would not generalize. The two approaches are, however, equivalent in the way described in the original question and in my answer. – Dave Dec 01 '21 at 16:43