1

I am conducting a study where I am randomly assigning people to different conditions. Because I am recruiting from a representative sample in the US, there will be some variability in the demographics (although not too much since random assignment takes care of that).

Is it possible to test that the variability in the demographics (eg. ethnicity, sex, age) between the different samples don’t matter in my results? I will be correlating the measures I collect from the participants. If so, how?

Thanks!

keji11
  • 87
  • Do you keep track of the demographics that you are interested to check for balance between conditions? – dipetkov May 12 '22 at 21:50
  • @dipetkov Hi! yes I can check the demographics of the samples as they roll in. However, I would not be able to assign a specific condition to each participant based on their demographics. So…I am in effect trusting random assignment to do most of the work for me. Is there a way to test that the slight demographic variability in the different conditions does not matter OR control for these demographic variables in the correlation? – keji11 May 12 '22 at 22:04

1 Answers1

1

You can run tests to check that demographic variables are balanced. However, it's not recommended to do so for a randomized study. As you say, you have to trust the randomization. What's recommended instead is to include the demographic variables as covariates in your analyses, esp. if the covariates are strong predictors of the outcome.

If you want to check that the randomization procedure is implemented correctly, it's better to examine this directly rather than through a test. Think about it this way: What are you going to do if the statistical test for covariate balance between conditions rejects the null hypothesis? Either the randomization wasn't implemented correctly or the imbalance occurred by chance. The test cannot tell you which one of these things is more likely to have occurred.

References:

Baseline Balance and Valid Statistical Analyses: Common Misunderstandings
Balance checking in randomized controlled trials with large sample

dipetkov
  • 9,805