2

Given I have a binary outcome (Y), binary exposure (A), binary potential effect modifier (Z), and a bunch of covariates (C1-C20).

We would like to examine for effect modification of Z on the treatment effect of A on Y. The treated vs. not treated have different characteristics and we plan to either weight or match treated/not treated using a propensity score.

My question is whether the effect modifier should be included in the propensity model (P(A|C1-C20,Z))? Eeren et al. (2015) seem to indicate no, but I've found little else to back that up. If not, what is the best way to test for effect modification, while still balancing covariates using a propensity score without the effect modifier (P(A|C1-C20))? Possible methods I see:

  1. Make two propensity models within each effect modifier group. Then look at treatment effect within each of these subgroups.
  2. Make single propensity model excluding effect modifier. Then include effect modifier into effectiveness model as an interaction term with treatment.

Any suggestions with references to published papers would be appreciated.

LeslieKish
  • 238

2 Answers2

1

In general, you should include all pre-treatment causes of the outcome in the propensity score model, even if they don't cause selection into treatment. That said, the answer to your question is less about the propensity score model itself and more about its effect on the resulting samples.

For moderation analysis, you must ensure balance is achieved on the covariates within each level of the moderator. Because the goal of moderation analysis is to compare two (or more) effect estimates, you need to ensure each of those estimates is free from bias due to imbalance. Often, the best way to do this is by estimating subgroup-specific propensity scores (equivalent to including interactions between the moderator and every covariate in the propensity score model if it is a parametric model). It might be possible to let a machine learning algorithm like GBM or BART automatically decide whether moderator-covariate interactions are required.

There is also a method called the subgroup balancing propensity score (SBPS) developed by Dong et al. (2020) that attempts to decide whether an overall propensity score or a subgroup-specific propensity score yields the best balance for each subgroup. For propensity score weighting, this is implemented in R using the sbps() function in the WeightIt package.

For more reading on this matter, I recommend Dong et al. (2020) and Green and Stuart (2014).

If you are doing matching, you should perform exact matching on the moderator. If you are doing weighting, you should ideally estimate weights seperately within each level of the moderator, but this isn't always necessary if you can achieve subgroup-specific balance without doing so.

To estimate the treatment effect, fit a regression of the outcome on the treatment, the moderator, and their interaction in the matched or weighted sample (i.e., including the matching or propensity score weights). You can use regression contrasts or different parameterizations of the model to get subgroup-specific effects, an overall marginal effect, and tests for moderation. See the section in the MatchIt vignette on moderation analysis for more details.

Noah
  • 33,180
  • 3
  • 47
  • 105
  • I would reverse your first sentence. Based on many sources including Rubin’s 2008 paper, I've concluded that the covariates used to create propensity scores should be any that predict treatment/no treatment – irrespective of any relationship that covariates may have with the outcome. (Except of course we would not include parallel measures of Y, as this would distort, "muddy," or "cannibalize" relationships.) – rolando2 Dec 01 '22 at 17:17
  • @rolando2 "Many sources" except the canonical reference for variable selection in propensity score models, Brookhart et al. (2006)? Variable Selection for Propensity Score Models -- "The results suggest that variables that are unrelated to the exposure but related to the outcome should always be included in a PS model. ... In very small studies, the inclusion of variables that are strongly related to the exposure but only weakly related to the outcome can be detrimental to an estimate in a mean squared error sense." – Noah Dec 01 '22 at 20:06
  • That may well be a good source, but so much has been done since 2006 -- by Rubin, VanderWheele, Shpitser, Griffin, many others. Also, when I wrote "I would reverse...." I was telling my best sense of the topic; it's not that I was so sure. I think the best course is still quite debatable. – rolando2 Dec 04 '22 at 21:57
0

The specific answer is no, Z should not be included in the creation of a PS. Also, if you have a bunch of covariates c1 - c20, perhaps a DAG can clarify the minimal sufficient adjustment set for estimating the direct effect of A on Y and then perhaps covariate adjustment might be possible. PS methods are agnostic to both HTE and effect modification and are best avoided if feasible. A logistic regression with A, Z and A#Z balanced for potential covariates is still not very useful because there is no way to determine if a potential non-null product term in this GLM is due to an artifact of the sample (overfitting) or to true effect modification by Z. See this paper

Suhail
  • 3