1

I'm following this example (see below for the source):

Breast Cancer No Breast Cancer Total
DDT exposed 360 13636 13996
Unexposed 1079 76313 77392 
RR = 1.87
CI = 1.66 - 2.10

What I do not like is this: if I multiply the number of "No Breast Cancer" by a large number, even one million or more, these two numbers do not change at all. But now the occurrence of the event is very rare and the difference from 360 to 1079 may well be due to random chance.

I'm surprised that the ratio between the positive samples and the whole population is not even considered when we compute the confidence interval.

Intuitively 360 out of 13636 is different from 360 out of 136,360,000 and I would expect the confidence in the measurement to reflect this by being much lower.

Am I missing something? Thanks.

EDIT: I'll try to explain it with an example:

First experiment. My sample is 1000, half of this is young, half is old. I do a random split, the worst case is to have a "shift" of 250 units from the average. Now I run my tests and I get 5 cases in group A and 25 in group B.

Second experiment, completely different and independent from the previous one. My sample is one million, half young, half old. I do a random split. I think the average "shift" I can expect from the split is very high, I think it is possible to compute it, probably higher than 250. Now I run my tests and I get 5 cases in group A and 25 in group B, just like before.

Isn't the "signal" in the second experiment too weak to conclude something with the same "confidence" of the first one?

Source:

Please note that the numbers are slightly different from the ones in the table in the original example but match the one used in the formula.

"Confidence Intervals for the Risk Ratio (Relative Risk)"

"A Nested Case-Control Study"

lorenzo
  • 131
  • Re "The difference from 360 to 1079 may well be due to random chance." This is incorrect: perhaps that misconception is the basis for your question? – whuber May 24 '21 at 15:21
  • @whuber Why? Let's say I'm doing a trial splitting my population randomly in half. If the population is very large I would expect to have a large absolute source of error or "noise" from the randomness of the split itself, that might even be bigger than the effect I'm trying to measure. If the population is small, the probabilty of an "unbalance" of the same magnitude is smaller if not impossibile. – lorenzo May 24 '21 at 16:45
  • That's a common intuition -- but it's incorrect. The sampling uncertainty is essentially unrelated to the population size, unless the population is less than about ten times the sample. I provide some guidance at https://stats.stackexchange.com/a/31571/919. – whuber May 24 '21 at 16:51
  • @whuber Sorry, the post did not help me. I updated my question with an example. – lorenzo May 24 '21 at 19:01

1 Answers1

2

The "relative risk" of 1.86 is actually an odds ratio, calculated by $A \times D / (B \times C) =$360 * 76313 / (13636 * 1079). Notably, entries B and D are not a denominator, they are non-cases, so if you multiply non-cases by some constant, your OR remains unchanged but the RR will change.

Now your confusion seems to come from the idea that you've been told that the odds ratio approximates the relative risk when the outcome is "rare". So you are asking, what happens when, instead of tens of cases, you have hundreds or thousands of cases. Isn't the outcome no longer "rare"?

The answer is that this "rareness" has to do with the prevalence of cases in the population or sampling frame rather than the overall number of cases in the sample. Suppose for instance, I perform a case control study of children diagnosed with Hunter's syndrome among a hospital network in a small region of the US to model the association with family history as a motivator to perform a family-based study. There might be 10 such kids over a one year retrospective review, and an odds ratio of 5.4 with any family history. Then I partner with a clinical trial network and I identify over 1,000 such kids with over 1,000 well-matched controls. The odds ratio is still 5.4, and it still approximates the risk ratio, to provide strong evidence of the heritability of the condition.

The variance of the CI will change in these conditions. For the log odds ratio, the variance is given by a nifty formula that you should memorize

$$ \text{var}(\log (OR)) = 1/A + 1/B + 1/C + 1/D$$

where $A, B, C, D$ are the cells in the 2 by 2 tabular array of outcomes. In other words, recruiting more patients will increase the precision of the estimates, leading to narrower CIs, and better power.

AdamO
  • 62,637
  • thanks for the long answer. In your conclusion you say that recruiting more persons improves the CI because you assume that this will also increase the occurrences and I agree. Instead I want to compare the "strength/quality" of two different experiments one with a small sample and one with a large one but with the same number of positives. The larger experiment seems weaker to me. I've updated my question with one example. If this is true, I expected to be able to see this from the CI but it is not the case. – lorenzo May 24 '21 at 19:23
  • 1
    @lorenzo your two experiments will not produce the same odds ratios because you're increasing the "non-cases" by a non-multiplicative amount. They'll be almost the same. You'll see some improvement by sampling 1,000,000 instead of 1,000. But actually write out the variance expression you have V=1/5 + 1/25 + 1/(big number) + 1/(big number). Make big number bigger won't give you THAT much more precision. – AdamO May 24 '21 at 19:29
  • if I multiply the "No Breast Cancer" column by 10000 I see no changes in RR and an extremely small difference in CI. But I consider the results of this new case to be much "weaker" than the original one, do you agree?. The positives are so small, 0.0003%, that just about anything could sway the result in real life. Variance is going to be smaller. – lorenzo May 24 '21 at 20:36