What is the lower and upper bound on $R^2$ with no intercept?

Question

With an intercept in linear regression, I know that $R^2$ is bounded by [0,1]. Without an intercept, I know that $R^2$ can be negative. What is the lower bound for this case? And is the upper bound still 1 for this case?

Answer 1

There is no lower bound.

Consider an intercept-free regression on two points, $(0,a)$ and $(1,a+2)$. By forcing the regression line to go through the origin, you can make the fit as bad as you want by making $a$ very large.

I think the upper bound remains $1$, but I confess that I’m not sure, though I find it hard to believe otherwise.

EDIT

The upper bound is $1$. Consider a regression through $(0,0)$, $(1,1)$, and $(2,2)$.

Answer 2

+1 to Dave's answer about the lower bound.

And yes, the upper bound is still $1$. In

$$ R^2 = 1-\frac{SS_\text{res}}{SS_\text{tot}}, $$

the numerator and the denominator of the fraction are both sums of squares, so they are nonnegative, so the entire fraction must be nonnegative. So the entire expression is maximized (with a value of $1$) if the fraction is zero.

And this will happen in a regression without an intercept if and only if all data points lie on a line that goes through the origin. So this upper bound will be achieved in a specific case.

Answer 3

In order to determine the lower/upper bound of the $R^2$ for a linear model $y = X\beta + \varepsilon$ without an intercept (i.e., $X$ does not contain the column $e$ of all ones), it is helpful to first review why $R^2 \in [0, 1]$ when $X$ contains $e$.

In matrix form, regardless $X$ contains $e$ or not, $R^2$ can be written as (the first equality is by definition, the second equality is by algebra) \begin{align*} R^2 = 1 - \frac{y^\top(I - H)y}{y^\top(I - n^{-1}ee^\top)y} = \frac{y^\top(H - n^{-1}ee^\top)y}{y^\top(I - n^{-1}ee^\top)y}, \tag{$\star$}\label{0} \end{align*} where $H = X(X^\top X)^{-1}X^\top$ is hat matrix.

When $e$ is a column of $X$, one can easily verify that $He = e$ (geometrically, this is very obvious: when you project a vector onto a space that contains this vector, you get the vector itself), whence the matrix $H - n^{-1}ee^\top$ is idempotent: \begin{align*} (H - n^{-1}ee^\top)^2 = H^2 - n^{-1}Hee^\top - n^{-1}ee^\top H + n^{-1}ee^\top = H - n^{-1}ee^\top. \end{align*} Together with that $H - n^{-1}ee^\top$ is symmetric, it follows that $$y^\top(H - n^{-1}ee^\top)y = \|(H - n^{-1}ee^\top)y\|^2 \geq 0. \tag{1}\label{1} $$ This shows that $R^2 \geq 0$. On the other hand, since \begin{align*} y^\top(I - n^{-1}ee^\top)y - y^\top(H - n^{-1}ee^\top)y = y^\top(I - H)y = \|(I - H)y\|^2 \geq 0 \end{align*} and $y^\top(I - n^{-1}ee^\top)y > 0$ always hold (unless $y$ is a multiple of $e$, for which case $y$ has no variability and we usually rule this case out), we see that $R^2 \leq 1$. Note that since $I - H$ is idempotent and symmetric regardless $X$ contains $e$ or not, this part of the proof also applies to the intercept-free case. That is, $1$ continues to be the upper bound of intercept-free $R^2$.

Now when $e$ is not a column of $X$, the key property $He = e$ in general no longer holds. Consequently, $\eqref{1}$ becomes invalid (i.e., $y^\top(H - n^{-1}ee^\top)y$ can be negative). However, as argued above, $y^\top(I - n^{-1}ee^\top)y$ is always positive. As a result, the right hand side of $\eqref{0}$ may become negative.

One step back, then does it have a fixed lower bound (possibly negative)? The answer is no. For example, consider the case $n = 2, p = 1$, $X = \begin{bmatrix}0 \\ 1 \end{bmatrix}$, $y = \begin{bmatrix} y_1 \\ y_2 \end{bmatrix}$. Direct evaluation of $\eqref{0}$ yields \begin{align*} R^2 = \frac{-(y_1 + y_2)^2 + 2y_2^2}{(y_1 - y_2)^2}. \end{align*} Setting $y_2 = M$, $y_1 = M + 1$ in the above expression gives \begin{align*} R^2 = -2M^2 - 4M - 1 \end{align*} which tends to $-\infty$ as $M \to \infty$. This shows that intercept-free $R^2$ cannot be bounded from below.