0

Adjusted R-squared is defined as

r2_adj = 1-[((1-r2)(n-1))/(n-k-1)]

I have read at many places that adjusted R-squared is strictly less than R-squared, because k>0 and R-squared <=1. But I was just thinking upon this formula and it seems that adjusted R-squared will be greater than R-squared if somehow my number of data points n is less than number of independent variables k.

Is this an basic assumption too that n should always be greater than k+1?

Utkarsh Prakash
  • 3
  • 1
    When you have fewer data points than variables, what is the value of the unadjusted $R^2$?? – whuber Mar 13 '21 at 20:34
  • Its not dependent on variables or data points right? Lets assume R2 is 0, i.e. model is always predicting mean for all the instances. – Utkarsh Prakash Mar 14 '21 at 04:05

1 Answers1

1

Note first that there are many different R-squared estimators. For an overview, see the references below. Disclaimer: I am the author of the third one.

However, all of them indeed require that $p>N$, as otherwise, it is impossible to fit the regression model, in the sense that no unique regression coefficients exist. If you for example try to fit a model with $11$ predictors and $10$ samples, using the lm function in R, it will not provides estimates for two predictors and also no standard errors for any coefficient.

While you technically can get an R-squared from such a model, it is guaranteed to be $1$ and thus meaningless, as will any adjustment of this value.See also the comment by whuber.

You can of course obtain a prediction function for the $p>N$ setting using a different technique, for example, ridge regression. However, all the adjusted R-squared formulas only work for $R^2$ estimates as obtained by ordinary least squared regression. You could fall back to predictive $R^2$ for this case but this estimate something else than adjusted-$R^2$, see my answer to another question: https://stats.stackexchange.com/a/518400/30495

References

$[1]$: Shieh G (2008): Improved shrinkage estimation of squared multiple correlation coefficient and squared cross-validity coefficient. Organizational Research Methods, 11(2): 387-407 (link)

$[2]$: Yin P, Fan X (2001): Estimating $R^2$ shrinkage in multiple regression: A comparison of different analytical methods. The Journal of Experimental Education, 69(2): 203-224 (link)

$[3]$: Karch J (2020): Improving on adjusted R-squared. Collabra: Psychology (2020) 6 (1): 45. (link)

Julian Karch
  • 1,890
  • 1
  • 18
  • 29
  • The problem is the opposite of your characterization: when the rank of the model matrix equals $n,$ there will be a perfect fit (and probably many of them), making $R^2=1,$ whence any formula of the form $1 - (1-R^2)*\text{positive constant}$ will give $1$ for its adjusted version. The attempt to divide by $n-k-1$ in the formula is a strong hint that using $R^2$ and attempting to adjust it in such circumstances makes no sense. – whuber Apr 06 '21 at 15:00
  • 1
    This just seems to be a misunderstanding in what I meant with it is impossible to fit the regression model. I edited for clarification. – Julian Karch Apr 06 '21 at 15:16