3

Let's say we want to compare two probabilities $p_1$ and $p_2$, not necessarily referred to the same population. For example, $p_1$ may be the probability of getting a certain disease conditioned on having been vaccinated, and $p_2$ the probability of getting the disease for non-vaccinated people.

Common measures to compare two probabilities are their risk difference $p_1-p_2$, relative risk $p_1/p_2$ and odds ratio $p_1(1-p_2)/(p_2(1-p_1))$.

Is there any setting or application field where the measure $p_1/(p_1+p_2)$ is used? I see it as a "normalized" version of $p_1/p_2$. It is similar to that if $p_1$ can be assumed to be significantly smaller than $p_2$, with the advantage that it is guaranteed to be bounded between $0$ and $1$, which can lead to certain estimation techniques.

Any ideas or pointers would be appreciated.

Luis Mendo
  • 1,069
  • One truly problematic aspect of this proposal is its extreme sensitivity to an arbitrary decision: namely, how to express the probability. When you state the same information in terms of the chances of the complementary events, those chances are $1-p_1$ and $1-p_2,$ resulting in $(1-p_1)/(2-p_1-p_2).$ The risk difference and odds ratio do not suffer from this deficiency, although the relative risk does. This points to the need to carefully delineate the application of any such metric. Most likely, for instance, you would insist on both $p_1$ and $p_2$ being tiny. – whuber Mar 24 '23 at 13:34
  • 1
    BTW, for an account of many, many more ways one might compare probabilities, see https://stats.stackexchange.com/a/201864/919. – whuber Mar 24 '23 at 13:35
  • 1
    @whuber Yes, what I have in mind is p1, p2 small, with one of them expected to be clearly smaller than the other, as in my example. Thank you for the link! – Luis Mendo Mar 24 '23 at 14:08

1 Answers1

1

You can certainly invent any number of metrics to compare two proportions. Since you mention risk difference, risk ratio, and odds ratio, common comparisons of independent samples, most would assume you are speaking about two sample test of proportions.

Other measures of association for binary endpoints include:

In a two sample test, your metric $\theta = p_1/(p_1+p_2)$ could be interpreted as the probability that a randomly selected "case" belongs to group A, and so if $\theta > 0.5$, there's an excess in group A. Not meaningful for case-control studies without weighting.

As an estimator, $\theta$ is almost certainly guaranteed to underperform the MLE, that is logistic regression and inference on the log odds ratio which is the natural parameter of a binomial random variable. The lack of boundedness is what makes logistic regression so powerful.

If the events are complementary as @whuber points out then, the test has a conditional intepretation and you can use conditional probability laws to identify their respective "estimators" and tests.

AdamO
  • 62,637
  • Thank you for your answer. Can you please explain what you mean by "group A", and how $\theta > 0.5$ implies an excess in group A? Also, when you say MLE, is that the MLE of $p_1/p_2$, or of something else? – Luis Mendo Mar 23 '23 at 23:26
  • note that in the question p1 and p2 didn't necessarily relate to the same population (but they could do). Also, this doesn't work so well if there is an intersection between the events that p1 and p2 correspond to (so that p1+p2 isn't the chance of "event 1 or event 2 happening" any more) – JDL Mar 24 '23 at 10:11
  • @LuisMendo you say p1, p2 are "not necessary defined on the same 'population'". So we assume this is a two-sample test. Please confirm. Also, the MLE can be derived exactly by writing out the likelihood for the two binomial experiments. Spoiler alert: logistic regression and the Pearson Chi-Sq test of independence are really good tools. – AdamO Mar 24 '23 at 13:11
  • @JDL agree, I'd liken this to asking, "I'm going to amputate a leg, it may be gangrenous or it may be healthy we can't be sure..." If "p1 and p2 'coming from the same population'" means they are complementary responses, you don't have binomial likelihoods but multinomial. You'd have to go through the exercise of writing out that likelihood and deriving different MLEs to get answers about "good estimators". – AdamO Mar 24 '23 at 13:13
  • @AdamO p1 and p2 may come from different populations, as in my example, or from the same. All I'm interested in knowing is if there is any scenario where this measure can be useful – Luis Mendo Mar 24 '23 at 14:06
  • 1
    Certainly there is, because when the events are disjoint, this measure is the conditional probability of event $1$ given that $1$ or $2$ occurred. – whuber Mar 24 '23 at 14:37
  • @LuisMendo you now have interpretations from both whuber and I, I think your question is addressed. – AdamO Mar 24 '23 at 15:16
  • @AdamO Sorry, I still don't understand what you mean by "group A". How is group A defined? – Luis Mendo Mar 24 '23 at 15:50