3

If error terms are correlated between observations, will that reduce how predictive a model is? Specifically, given the same predictor variables, will the mean square error of the model's predictions tend to be larger when the correlation between error terms in larger?

I understand how it increases the standard error of the coefficients of predictors, but I don't understand how it increases the uncertainty about predictions of a dependent variable. For example, suppose that I wanted the predict future values of some outcome. A simple prediction could be to guess the average. I don't see how correlated errors would reduce how accurate that prediction is. You're still getting another sample of the value of the outcome.

Why does it reduce predictive power, if it does? What is a simple example of why correlated error terms reduce predictive accuracy?

Would accounting for the correlation improve the model accuracy? How would one remove the correlation?

user7340
  • 403
  • 1
    You might be confusing your terminology, autocorrelation and correlation between error terms is not the same thing. – Bar Apr 09 '15 at 09:41

2 Answers2

2

As far as correlation between error terms you might be talking about serial correlation.

You can find a good resource here

Briefly, it will not affect the unbiasedness or consistency of OLS estimates, but it will affect the efficiency of the estimator.

What this means is that if you have positive correlation, your model will end up looking better (smaller variance in $\hat{\beta}$) than it actually is. That would mean that it is possible to obtain a better estimator than OLS (it is no longer a BLUE estimator)

See section 10.3 here for more details.

Bar
  • 2,862
  • I am talking about serial correlation of error terms. Will it affect the efficiency of estimates of everything, or just correlation coefficients? I'm interested purely in how efficient predictions of the value of the dependent variable are. – user7340 Apr 09 '15 at 13:25
  • @user7340 You can check out my updated answer. The short of it is that your model will appear to be better than it really is, and that you can do better than OLS. – Bar Apr 09 '15 at 13:55
  • Appear better for coefficients I get. I'm only interested in predictive accuracy. – user7340 Apr 09 '15 at 14:44
  • FTA: "Forecasts based on OLS estimated will be unbiased. But forecasts are inefficient with larger variances." – Bar Apr 09 '15 at 14:52
  • Thanks. Do you have any intuition for that? I understand how correlated observations would make forecasts inefficient. If knowing something about one observation tells me something about another observation, that's worth less than having two independent pieces of information. However, if error terms are correlated that still leaves the non-stochastic component of draws from a random variable uncorrelated. Is the idea that even if the non-random component is independent, the random component is not and so the value of draws are partially correlated? – user7340 Apr 10 '15 at 04:43
  • You can get a good intuition from this Q&A: http://stats.stackexchange.com/q/26133/16052 – Bar Apr 10 '15 at 08:27
0

Yes it will tend to make it worse because you are effectively limiting the amount of data your statistical model gets to use. Think about the variance-bias tradeoff. Even if bias is still 0, the variance will increase if you have correlated data.

convolutedstatistic
  • 213