3

When applying the Bayesian information criterion, one has to use an "effective sample size" in the penalty term. E.g. if observing longitudinal data (e.g. changes in the blood pressure of an individual over time), the sample size to use should be somewhere between the number of observed individuals and the number of measured values. How to calculate this in general is not clear.

My question is: can the effective sample size depend on the model? Does it mean that BIC can be reliably used only with nested models, as then the effective sample size would be constant?

quant_dev
  • 654
  • 4
  • 13

1 Answers1

3

It definitely depends on the model. Think e.g. of correlated multivariate normal observations (e.g. a MMRM). A model that assumes a diagonal covariance matrix is nested within one that allows for a completely unstructured covariance matrix. In the former case the effective sample size is the number of observations, in the latter case some number between that and the sample size, just as you said. Of course this is also a counter-example about whether the effective sample size is constant across nested models.

Björn
  • 32,022
  • 1
    Doesn't it subvert the idea of using BIC to select the "best" model? – quant_dev Sep 25 '15 at 11:24
  • That's also true (unless the BIC difference between "best" and second best model is really large, say, >15 and when there are very few similarly likely candidate models so that there is hardly any model uncertainty ), but perhaps a different topic. – Björn Sep 25 '15 at 13:22