Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [aic]

AIC stands for the Akaike Information Criterion, which is one technique used to select the best model from a class of models using a penalized likelihood. A smaller AIC implies a better model.

$\mathit{AIC} = 2k - 2\ln(L)$

where $k$ is the number of parameters in the statistical model, and $L$ is the maximized value of the likelihood function for the estimated model.

${AICc} = AIC + \frac{2k(k + 1)}{n - k - 1}$

AICc is AIC with a correction for finite sample sizes, where $n$ denotes the sample size. Thus, AICc is AIC with a greater penalty for extra parameters.

AIC was introduced by Hirotugu Akaike in his seminal 1973 paper "Information Theory and an Extension of the Maximum Likelihood Principle" (in: B. N. Petrov and F. Csaki, eds., 2nd International Symposium on Information Theory, Akademia Kiado, Budapest, pp. 267{281).

References:

Wikipedia

"Information Theory and an Extension of the Maximum Likelihood Principle" (starts on page 610).

1029 questions
25
votes
3 answers

Meaning of 'number of parameters' in AIC

When computing AIC, $AIC = 2k - 2 ln L$ k means 'number of parameters'. But what counts as a parameter? So for example in the model $y = ax + b$ Are a and b always counted as parameters? What if I don't care about the value of the intercept, can…
Sideshow Bob
  • 1,485
15
votes
1 answer

Is AICc ever worse than AIC?

I know that when the number of observations is small (one paper says <40 times the number of parameters), AICc should be used instead of AIC for model comparison. Does this imply that when the number of observations is large, AIC should be used…
user21412
  • 277
  • 1
  • 2
  • 8
9
votes
1 answer

What breaks the comparibility of models with respect to the AIC?

Supposing I have fit some models using predictors (and the response variable) from the same data set. What changes to the model will make it unreasonable for me to compare the models on the basis of AIC? 1) Supposing, if I log transform the…
Ali Turab Lotia
  • 649
9
votes
1 answer

On Negative AIC Values

My question is related to the thread Negative values for AIC in General Mixed Model. I often get negative AIC values from the software I use. I notice it most when I'm doing time series. But here is what I don't get. When defining the AIC like…
Zachary Blumenfeld
  • 3,974
7
votes
1 answer

Comparing differences of AIC of different data sets

Let's assume we have two linear regression models $m_1$ and $m_2$, where $m_2$ is nested in $m_1$, and two data sets $d_1$ and $d_2$ which are of different size. Calculating the AIC for each pair shows that the following equation is true: $$…
Funkwecker
  • 3,082
6
votes
4 answers

What is a difference between a Low AIC and a Bigger AIC

I inputted my data sets on r and it spit 2 AIC, one with interactions and one without it. Without Interactions I got 682.4, and with interactions I got an AIC of 684, the difference is minimal, but I do want to understand what does that mean.
Killercamin
  • 215
5
votes
3 answers

Can I compare AIC values of a linear function with a non-linear function?

Can I compare AIC values of a linear function with a non-linear function? Because I get totally different results. One is 4000 other the 6000000. Estimation is done on the same data setvariables.
user2298382
  • 75
5
votes
1 answer

A problem with two ways of calculating AIC

I find that there are two ways to calculate AIC: AIC = -2ln(likelihood)+ 2K and AIC = n*ln(RSS/n)+2K I have: crf <- c(0.3333333, 0.5000000, 0.6666667, 0.6666667, 0.6666667, 0.8333333, 0.1666667, 0.3333333, 0.5000000, 0.5000000, 0.8333333,…
Tali
  • 163
4
votes
0 answers

AIC & number of parameters

I have two questions about AICc and number of parameters. I am using AICc to select the best of three models in a logistic regression - and each model has the same number of parameters. What are some compelling arguments for this method? I have G…
Brian
  • 41
3
votes
1 answer

Why Is AIC computed with a term containing -log(SSR) instead of -SSR?

Looking at https://en.wikipedia.org/wiki/Akaike_information_criterion I find the well known log likelihood $\ln\mathcal{L}(\mu,\sigma) \, = \, -\frac{n}{2}\ln(2\pi) - \frac{n}{2}\ln\sigma^2 - \frac{1}{2\sigma^2}\sum_{i=1}^n (x_i-\mu)^2$ for…
Ggjj11
  • 1,237
3
votes
1 answer

Calculating AIC

I have following data Model 1: Y ∼ B1 + B2 + B3 + B4: log L = −213.4 Model 2: Y ∼ B1 + B2 + B3 + B5: log L = −567.1 The question that i have is "Calculate AIC for these 2 models". It doesn't say much. I know the formula for AIC is $$ AIC = -2log(L)…
Em Ae
  • 351
3
votes
1 answer

What percentage decrease in AIC is significant?

I have two models with AIC values 2500 and 2590. So.... Is that significant? What percentage decrease is usually considered significant?
MICHAEL
  • 51
3
votes
1 answer

How exactly does AIC penalize overfitting?

I've been reading up alot on the AIC value for GLM models and it has come to my attention that pretty much all of my litterature claims that AIC penalizes the model with too many variables without mensioning what the penalty actually is. Is there…
Janono
  • 443
2
votes
1 answer

Combining AIC Scores

I am trying to combine two AIC scores from different parts of a single data set. I have two sigmoidal models that each predict about half of the data (imagine two sigmoidal curves attached end to end), each with a different AIC. I am interested in…
Alaksa2015
  • 31
  • 3
2
votes
0 answers

How to understand the AIC formula in matlab

What I'm acquainted to is the AIC formula given in wikipedia, that is AIC=2k-2ln(L) where k is the number of parameters and L is the maximized likelihood for a given model. Whereas the AIC given in Matlab (see here) is…
Yimai
  • 21
1
2