3

I want to test whether a treatment (=1 if treated, else 0) have effect on participant $i$'s outcome $Y$. The coefficient $\beta_1$ is significant in the main model: $$Y_i=\beta_1 treatment_{i}+\beta_3 sex_{i}+controls_{i}$$ But the coefficient $\beta_1$ becomes insignificant after including a interaction term between treatment and individual's gender(=1 if female, else 0):$$Y_i=\beta_1 treatment_{i}+\beta_2 treatment_{i}*sex_{i}+\beta_3 sex_{i}+Controls_{i}$$ The results is both $\beta_1$ and $\beta_2$ are insignificant, and $\beta_2$ is very small. Also, $\beta_3$ is small and insignificant in both models. I think this means there aren't much differences between treament's effect on male or female. But what I don't get is why are they not significant, but significant when combing together?

Key
  • 51
  • 4
  • Is sex dummy coded (0 = male, 1 = female)? If so the beta1 parameter should reflect the treatment effect for males only (i.e., a simple effect of treatment), whereas in model 1 its the effect across both genders (i.e. the main effect of treatment). If you use sum-coding on your gender variable (e.g., 0.5 = male, -0.5 = female) the treatment parameter should be comparable across both models. – AndreasM Nov 13 '20 at 22:13
  • Also, keep in mind that your should compare confidence intervals, not just the significance. – Ott Toomet Nov 14 '20 at 01:09
  • Assume for simplicity that treatment effect is 0. If you are not controlling for sex in the first model (which your notation suggest), then you could imagine a situation in which all males and only males where treated, so any effect of being male would come up as an effect of treatment. When you then add control for sex, the effect from the treatment and interaction for sex and treatment would become 0 (ex hypotesi of no treatment effect.) – Jesper for President Nov 14 '20 at 10:03
  • Don't speak of "significant" or "insignificant". These notions mean very little and are entirely arbitrary. Form contrasts of interest and look at their point estimates and uncertainty intervals. – Frank Harrell Nov 14 '20 at 12:40
  • Power is one likely reason why the simple effects for each gender come out "non-significant" . For the main effect you have a larger sample (males + females) and thus more power than for the simple effects for each gender alone. – AndreasM Nov 14 '20 at 13:16

0 Answers0