Why don't economists do regression diagnostics?

Question

There is a lot of talk about regression diagnostics in tutorials on the web, but then in economics research papers nobody actually reports residual plots, collinearity checks etc. Is there any reason for this?

Answer 1

To start with the question in the title:

Why don't economists do regression diagnostics?

People do regression diagnostics. I don't know of any respectable researcher that would not perform regression diagnostics and virtually any paper will have hints that the regression diagnostics was performed. For example, in tables with regression results you will find remarks on White or HAC errors being used to either correct for heteroskedasticity or autocorrelation or both, or references to corrections for cross-sectional dependence etc.

Only very unscrupulous scholars would make claims about these issues being/not being present without some testing. So scientists (or at least the good ones) always perform regression diagnostics.

To address the question in the body:

in economics research papers nobody actually reports residual plots, collinearity checks etc. Is there any reason for this?

Yes, almost all scientific journals have very strict page limit of between 30-60 pages with most journals having page limit around 40 pages. In addition shorter articles are often more preferred and attract wider readership because people usually prefer to read shorter papers. Also, note that page limits are usually inclusive of list of references which often can eat up another 1-5 pages and also all other stuff. Only online appendices are excluded from page limit.

Now documentation of regression diagnostic can easily eat up 10 pages if you want to do it properly with all the plots (or even more). Moreover, regression diagnostic is not of much interest in itself. You preform it in order to know how to properly specify your model or what identification strategy to use. Once you figure that you just use the appropriate model, so by itself regression diagnostic has little value for reader as it carries very little information about research result. As mentioned in the first part of the answers people will still mention in their paper that there was autocorrelation or heteroskedasticity and how they corrected for it (and so on for other problems), so there is not that much point in additionally wasting space in the paper on showcasing all the auxiliary diagnostics. Any mistrustful researchers can just request their data and rerun that diagnostic themselves.

Consequently, the reason for that is simply that there is not enough space for it in the paper, and because you always have to economize on the space given. If you would really want to do it, it would usually end up eating 1/3 of a precious space, You would be surprised how common problem it is for researchers to actually fit their research in the page limit. Often you will be forced to relegate even main derivations to online appendices just to fit in the limit. In the end anything that is not of great importance to support or interpret the main result will simply not make the cut.

As mentioned in the comments this can sometimes cause issues with reproducibility, but nowadays the solution to that is that journals require scholars to post their code that was used to derive results (where you would find also regression diagnostics) rather than actually report it in the paper, for the reasons mentioned above.

Answer 2

This is a very thoughtful question.

I think it is related with (i) the purpose and (ii) the sample size. Econometrics is very often concerned with causality (rather than prediction or forecast). For causality, correct model specification, consistency, and valid standard errors are important. Things such as multicollinearity (high correlation), non-normality, etc., are irrelevant (especially with large data sets).

For example, multicollinearity typically leads to large standard errors but, importantly, no biases. If you drop some variables due to multicollinearity, it just means you fail to control for the variables you initially intended to control for; that is, your estimator is biased. Nonnormality check (e.g., normal Q-Q plot) is no important as long as the sample size is large due to the central limit theorem. Outliers are data points just like any other; who gave you the authority to omit them at your will? By dropping the 'outliers' you are just restricting the population in a fancy way; you only get the criticism that your estimator is biased. VIF? If you drop variables due to high VIF, it means you have an inconsistent (biased) estimator.

Selecting a model based on data is dangerous. It will be hard for you to defend your model chosen by lasso if you want to say something about causal effects unless you experiment with bleeding edge econometric techniques. Inferences are to be done for a given model (created by your thought), not a model suggested by statistics (i.e., by a computer).

These days we don't even care much about testing heteroskedasticity becase the sample size is large and we can always do HC inferences. Autocorrelation is not an isssue, as it only complicates standard errors, which you can fix by HAC.

If you are interested in prediction/forecast, those things might be useful more. But even for that, the said diagnostics are too old-fashioned. People have already moved on to lasso and other machine learning techniques. I think the said diagnostics might survive in (non-econometric) textbooks and tutorials, but will die out eventually in econometric practices. But if you have small samples, the story is different. It also happens that old things are found useful in completely different contexts. For example, IF's are very useful for computing standard errors.