2

For a response variable $y$ and predictor $x_0$, I have data for a number of additional variables $x_n$, $n = 1, ..., 7$. I would like to control for a confounder in my GLM, let's call it "size". $x_1$, $x_2$, $x_3$ are all variables that measure "size" in a certain way, e.g. number of incidents, local volume of incidents, global volume of incidents. How do I treat these three variables, do I include them all, or just one of them? Should I include interaction terms, if so, just $x_1*x_2*x_3$, or also $x_1*x_2 + x_1*x_3 + x_2*x_3$? I am conscious not to construct an overly complicated model. Should I first check for relationships between each variable, in isolation? I.e. does $x_2$ actually increase with $x_3$.

I also have additional confounders, e.g. age, that are expected to vary with size. Does this necessitate further interaction terms between those two confounders? I am not sure if ending up with a 20-term GLM is a good plan... maybe I have misunderstood something. Thanks.

Samalama
  • 121
  • How strongly correlated are the 3 pairs? – mkt Sep 23 '22 at 09:54
  • Is this something I should measure, like by regressing $x_1$ vs $x_2$, etc.? – Samalama Sep 23 '22 at 10:24
  • Just the correlation coefficient. cor(x1, x2) if you use R. – mkt Sep 23 '22 at 10:26
  • 1
    If they are highly correlated and measure the same construct, I would think that doing a PCA on the 3 measures of size would be a simple solution. It would probably capture most of the variation in 1 dimension that you could use in your regression. – mkt Sep 23 '22 at 10:28
  • This sounds like a smart idea, thanks. I'm in Matlab at the moment, so presumably corrcoef(x1, x2) would be equivalent, and I'd conclude that if the off diagonals are, perhaps > 0.95, x1, x2, are highly correlated and I can reduce their dimensionality. – Samalama Sep 23 '22 at 11:16
  • 0.95 is a VERY high threshold; I'd be concerned at much lower ones. There are still a couple of potential concerns here, so if you can add the correlation coefficient values and (even more useful) plots of the pairwise relationships, I/we will be able to offer more specific advice. – mkt Sep 23 '22 at 14:28
  • I suggest asking your final point about additional confounders as a separate question. Again, more details about your data would be valuable for addressing that. – mkt Sep 25 '22 at 12:19

2 Answers2

1

If you want to estimate the total causal effect of $x_0$ on the outcome $y$, you have to control for all the confounders. So you are on the right track.

If you are presuming a linear model, you need to include into the linear model formula all the confounders as linear terms. If you have confounders that deterministically depend on other confounders (e.g. age), e.g. age $= x_1 + x_2$, then you don't have to (and shouldn't) include the dependent confounder (age).

For obtaining the linear effect of $x_0$ on $y$ you don't include extra terms like interaction terms between the confounders. You just have to make sure that when you fix all the terms in the formula other than the treatment and the outcome, all confounders have fixed values.

Note that you should not try to interpret the coefficients of the $x_i, i\ne 0$, this would be what they call the Table 2 Fallacy.

In summary, for your situation, presuming that age depends deterministically on the other confounders, the formula $$ y \sim x_0 + x_1 + x_2 + x_3 $$ suffices, and if age is only partially determined by the $x_i$, then you should use: $$ y \sim x_0 + x_1 + x_2 + x_3 + \mbox{age}. $$

frank
  • 10,797
  • This is very helpful, also highlighting the issue of table 2 fallacy I was unaware of. Many thanks. I don't think any confounders depend on each other deterministically, so I should include all of them; this makes sense. However, I don't quite understand in which scenario I would want to introduce interaction terms? This resource for example discusses interaction as: "2 variables interact when one influences the effect of the other", which I thought may apply to my GLM. – Samalama Sep 23 '22 at 10:30
  • 1
    Interaction terms are nonlinear terms that are quite common so they got an extra name. E.g. if you had an additional binary variable b and you suspected the linear relation between $x$ and $y$ to depend on b, i.e. $y = f_b x$ with $f_b$ different for different b, you would use the formula $y\sim x + b:x$ to learn the dependence of $f_b$ on b. – frank Sep 23 '22 at 10:41
0

When you have highly correlated predictors (termed ), it leads to problems in estimation and inference - though not prediction. So if your goal is estimation or inference, including all 3 predictors in the model without additional steps is probably not the best solution.

Because your situation involves three measures of the same construct (size), one simple solution to avoid the multicollinearity problem is to construct a single variable out of the three. Depending on the nature of the relationships, you may be able to simply do a PCA on the 3 measures. The first principal component would probably capture most of the variation in a single dimension that you could use in your regression.

I would advise first plotting the three pairwise relationships to examine the nature of the relationship and consider whether this single variable would be meaningful in your case. You could also consider whether the second and third principal components would be useful. Including all 3 principal components will not cause any multicollinearity problems (because they are orthogonal), but they are less likely to contain useful information.

mkt
  • 18,245
  • 11
  • 73
  • 172