3

I have analyzed some secondary data that relates to Plasmodium infection at three forest sites: inside the forest, at the forest fringe, and outside the forest.

The outcome variable is infection with Plasmodium parasites. I have about 60 variables (individual and household variables) of interest that I have analysed for their association with individual infection. I also have data on the households in which each person lives.

A previous study analyzed the pooled data and used random effects for households and villages (there were 16 villages in total: 9 outside the forest, 3 at the fringe, and 4 inside the forest). They included "forest proximity" as a fixed effect.

My MSc thesis has focused on the "inside-the-forest" sites only (there is strong evidence to suggest investigating the risk factors for inside-the-forest sites separately). I have only used a household random effect.

Can a variable (for example, sleeping outdoors) be statistically significantly associated with infection in the pooled data with random effects for household and village (all 16 villages), but not significant in any of the separate strata (with only household random effects)? My gut feeling is it can (and I certainly hope so), but I am still somewhat of a statistics novice.

If it can, is this just statistically logical, or would I be better off providing a reference to support this? If so, can anyone point me in the direction of an appropriate reference?

Trypanosoma
  • 171

1 Answers1

4

Your main question here is:

Can a variable (for example, sleeping outdoors) be statistically significantly associated with infection in the pooled data with random effects for household and village (all 17 villages), but not significant in any of the separate strata (with only household random effects)?

The answer is a clear yes, as this is a classic case of Simpson's paradox, where an association changes based off a confounding influence of another variable. On a secondary note:

The outcome variable is infection with Plasmodium parasites. I have about 60 variables (individual and household variables) of interest that I have analysed for their association with individual infection.

I'm not sure what model you are fitting, but I would suggest using a much more parsimonious model if possible. I'm guessing you have a basic hypothesis which better suits the modeling you are seeking to achieve that doesn't necessarily require so many variables, at least with respect to fitting a singular model.

Edit

As per Dipetkov's comments, there are additional contributions which may play a role. Binning data necessarily comes at a loss of information and a change in the error present in the model. Statistical significance also does not convey that an effect is present, nor that it is practically important (see example here).

Shawn Hemelstrand
  • 13,543
  • 2
    You may have just saved my thesis; thank you @Shawn! My final MLM only has a few variables in it. The 60 were the original variables first analysed in univariate models prior to selection for MLM. – Trypanosoma Mar 31 '24 at 11:10
  • 2
    Simpson's paradox is one possibility but doesn't it refer to a change in the direction of association specifically? Another explanation (more likely?) is that chopping up the data into 17 subset means that the residual error in each subset analysis is quite a bit higher than in the full analysis. – dipetkov Mar 31 '24 at 11:30
  • 1
    Also there is some indication that the OP is interpreting that stat. significance mean the effect exists and no stat. significance means the effect doesn't exists (a common but incorrect interpretation of statistical significance). – dipetkov Mar 31 '24 at 11:30
  • @dipektov Interesting point, I let that slip. Thank you. Also, from Shawn's link to Simpson's Paradox, I am interpreting the information there to suggest that my example is an example of Simpson's second paradox. – Trypanosoma Mar 31 '24 at 11:43
  • 1
    @dipetkov your points are spot on and I have edited for clarity. – Shawn Hemelstrand Mar 31 '24 at 11:48