6

What is the term $K$ in Akaike information criterion? The AIC is defined as $2K-2log(L)$, where $L$ is the maximized value of the likelihood function for the estimated model.

On the internet, I found three competing candidates:

  1. Number of parameters + error term (for simple linear one-predictor model, intercept, slope and error term: $K=3$)
  2. Number of parameters (for the linear one-predictor model, intercept and slope: $K=2$)
  3. Number of predictors (for the linear one-predictor model, the slope: $K=1$)

Which one is correct and why?

chl
  • 53,725
Timo J. Marjomäki
  • 61

1 Answers1

4

This is how the original 1974 paper by Hirotugu Akaike defines the AIC:

AIC = (-2)log(maximum likelihood) + 2(number of independently adjusted parameters within the model)

The error term is not a parameter which you're independently trying to adjust, but the intercept is (e.g. your slope might be zero and the data best fit by a horizontal line). The correct answer for your simple univariate regression is $K=2$ (intercept and slope).

Cristian
  • 1,108
  • "The error term is not a parameter which you're independently trying to adjust, but the intercept is (e.g. your data might best be fit by a horizontal line)." I think you mean "...fit by a line through the origin," since omitting the intercept term constrains the intercept to be zero, but yields a line with slope $\beta$. – Sycorax Sep 26 '13 at 13:34
  • The point I was trying to make was that the intercept is an independently adjusted parameter. That is, you could have a slope of zero, and an intercept of, say, one. This would draw a straight line through the data. Or you can start with a slope $\beta$ and then adjust the intercept until you get the best fit. The point is that the intercept should be counted. – Cristian Sep 26 '13 at 13:38
  • I agree that the intercept is an independently-adjusted parameter. The sentence seems to indicate that the intercept parameter determines whether or not the line is horizontal, which is not the case. – Sycorax Sep 26 '13 at 13:47
  • Thanks for the quick answer. I was just wondering whether we must add a variance component to the estimated parameters (the first one above with error term perhaps implies that), bacause it has been counted in in some cases e.g. http://www4.ncsu.edu/~shu3/Presentation/AIC.pdf http://stats.stackexchange.com/questions/67786/aic-number-of-parameters – Timo J. Marjomäki Oct 02 '13 at 08:36
  • So in the multivariate case, would we increment K by 1 for each column/attribute added to the regression model, after the initial one? This is a great answer, but I want to make sure I understand it clearly; asking a separate question would probably qualify as a duplicate. – SQLServerSteve May 20 '17 at 00:05