Using R^2 in nonlinear regression

Question

This page discusses why Minitab does not compute $R^2$ for nonlinear regression. I understand that calculating $R^2$ between the response and the predictor ($y$ vs $x$) is not justified. However, is there any reason why calculating $R^2$ between the response and the predicted response ($y$ vs $\hat{y}$) is invalid?

(I know there are other goodness-of-fit metrics that may be better suited for nonlinear regression, but in this case I'm interested in $R^2$.)

Answer 1

It depends on what is meant by $R^2$. In simple settings, multiple definitions give equal values.

1. Squared correlation between the feature and outcome, $(\text{corr}(x,y))^2$, at least for simple linear regression with just one feature

2. Squared correlation between the true and predicted outcomes, $(\text{corr}(y,\hat y))^2$

3. A comparison of model performance, in terms of square loss (sum or squares errors), to the performance of a model that predicts $\bar y$ every time

4. The proportion of variance in $y$ that is explained by the regression

In more complicated settings, these are not all equal. Thus, it is not clear what constitutes the calculation of $R^2$ in such a situation.

I would say that #1 does not make sense unless we are interested in a linear model between two variables. However, that leaves the second option as viable. Unfortunately, this correlation need not have much to do with how close the predictions are to the true values. For instance, whether you predict the exactly correct values or always predict high (or low) by the same amount, this correlation will be perfect, such as $y = (1,2,3)$ yet $\hat y = (101, 102, 103)$. That such egregiously poor performance can be missed by this statistic makes it of questionable utility for model evaluation (though it might be useful to flag a model as having some kind of systemic bias that can be corrected). When we use a linear model fit with OLS (and use an intercept), such in-sample predictions cannot happen. When we deviate from such a setting, all bets are off.

However, Minitab appears to take the stance that $R^2$ is calculated according to idea #3.

$$ R^2=1-\left(\dfrac{ \overset{N}{\underset{i=1}{\sum}}\left( y_i-\hat y_i \right)^2 }{ \overset{N}{\underset{i=1}{\sum}}\left( y_i-\bar y \right)^2 }\right) $$

(This could be argued to be the Efron pseudo $R^2$ mentioned in the comments.)

This means that Minitab takes the stance, with which I agree, that $R^2$ is a function of the sum of squared errors, which is a typical optimization criterion for fitting the parameters of a nonlinear regression. Consequently, any criticism of $R^2$ is also a criticism of SSE, MSE, and RMSE.

I totally disagree with the following Minitab comment.

As you can see, the underlying assumptions for R-squared aren’t true for nonlinear regression.

I assumed nothing to give the above formula except that we are interested in estimating conditional means and use square loss to measure the pain of missing. You can go through the decomposition of the total sum of squares (denominator) to give the "proportion of variance explained" interpretation in the linear OLS setting (with an intercept), sure, but you do not have to.

Consequently, I totally disagree with Minitab on this.

Answer 2

Taking the other side: The $R^2$ in OLS has a number of definitions and interpretations that are endemic to OLS. For instance, a "perfect fit" has $R^2 = 1$ and, conversely, a "worthless" fit has $R^2 = 0$. In OLS the $R^2$ is interpreted as a "proportion of 'explained' variance" in the response. It also has the formula $1 - SSR/SST$

You say "non-linear regression" but I think you mean generalized linear models. These are heteroscedastic models that not only transform the response variable, but also express the mean-variance relationship explicitly, such as in a Poisson regression where the variance of the response is proportional to the mean of the response. Contrast this with non-linear least squares where the $R^2$ continues to be a very useful metric.

So if we consider GLMs, none of the interpretations we enjoy about the $R^2$ are valid.

• A "perfect" fit will not necessarily perfectly predict all observations at every observed level. So, the theoretical upper bound may be some value less than 1. Adding a predictor to a model does not optimally improve the $R^2$ in terms of that predictor's contribution: non-linear least squares would do that.
• The probability model for a GLM does not invoke a "residual" per se, (or methods that do do not treat the residual as normally distributed). So neither does the formula make any sense nor can it be interpreted as a fraction of "explained" variance.
• While incremental increases in the $R^2$ indicate improved predictiveness, you can't be guaranteed of the scales or unit differences. For instance, if two candidate predictors $u,v$ increase $R^2$ by 5% when added as separate regressors in separate models, the first, $u$ may predict variance really well in the tails but overall be a very lousy predictor and have disappointingly non-significant resluts, whereas the second, $v$, may not appear to improve predictions much, but when accounting for areas with low variance, the overall contributions are substantially better and corroborate statistical significance.
• Applying $R^2$ in a GLM regardless is called a pseudo $R^2$.

In that regard, the GLM has a much more useful statistic, the deviance, which even R reports as a default model summary statistics. The deviance generalizes the residual for an OLS model, which has an identity link and gaussian variance structure. But for models such as Poisson the expression is:

$$ D = 2 (y \log y \hat{y}^{-1} - y - \hat{y})$$