4

I've had problems with this for years now.

The problem is: I do a multiple regression of some sorts on data that we have collected.

The data is usually something like, say, successful/unsuccessful outcome in a surgical patient and we are looking to see if a range of variables predict this.

Often times 'location' is a variable and so if we have 8 locations that were operated on and we are interested to see if any of these locations bear significantly worse outcomes than the rest I see only one way to do this:

Do eight regressions using each location as a base and then report on everything (massive p-hacking problem).

Surely there must be a simpler and more eloquent way?

Paze
  • 2,291

1 Answers1

1

Firstly, you should always fit a single model rather than using multiple different models for different variable outcomes. The case you are describing sounds like it would be a candidate for logistic regression, using the location variable as a categorical/factor input in the model (with any interaction terms you think are required). By doing this in a single model you will constraint the relationship of the other input variables to the outcome so that everything is estimated with a single data set. Of course, as with any regression analysis, this model is just a starting point for consideration, and you would look at diagnostic plots, etc., to ensure that you are satisfied with the model assumptions.

Once you have fit the model to the data, you can use an ANOVA comparison (between the full model and one without the location variable) to see if there is any evidence of a difference in prevalence between locations (see e.g., here). Testing for a difference between locations is logically equivalent to testing whether any location have a worse outcome than the rest. That comparison is a single test, so it does not entail any problem of multiple comparisons. However, if you want to go further, conducting hypothesis tests on individual locations, you will indeed have a multiple comparisons problem and you will need to take that into account in your comparisons. Cases of inference for multiple variables in regression are inherently complicated by the fact that uncertainty proliferates over the individual inferences. I recommend you follow standard approaches for dealing with "multiple comparisons" in this latter case.

Ben
  • 124,856