Effect size calculation for comparison between medians

Question

I have two groups of samples - disease and normal. I have calculated whether there exists any statistically significant difference between the medians of the two groups via the Mood's median test. The p-value results indicated that I have a significant difference with an $\alpha$ of 0.1. Now I want to calculate the effect size between the two groups. I was planning on using Cohen's d, which is given by $d=m_1-m_2/s_p$, where $m_1$ and $m_2$ are the means of the two groups and $s_p$ is the pooled standard deviations for the two groups. My question is whether I can modify this formula to calculate the effect size for the medians between two groups,where $m_1$,$m_2$ will be medians instead of means. Is it the right way to do that?

Answer 1

Cohen's d is probably not the best effect size statistic for a comparison of medians, since it is based on the mean and standard deviation(s), and is related to the calculations used in a t test.

I haven't seen this anywhere that I remember, but I suppose you could use an analogous statistic that uses the differences in medians and the median absolute deviation.

(median(B) - median(A)) / sqrt((mad(A)^2 + mad(B)^2)/2)

For normal samples, this will give a similar result to Cohen's d.

If I invented this, you can call it Mangiafico's d. :)

Of course, a simple difference in medians is also a useful effect size statistic. It isn't unit-less, but has units of the measurements. If you add confidence intervals, this is quite informative.

Finally, effect size statistics that might be used for a Wilcoxon-Mann-Whitney test would be useful: Vargha and Delaney’s A, Cliff’s delta, and the Glass rank biserial coefficient.

Addendum November 2022:

I added an implementation of the statistic that uses the differences in medians and the median absolute deviation to my rcompanion package. An example follows.

Group1 = c(0,  4,  8, 12, 16, 1,  5,  9, 13, 17,  2,  6, 10, 14, 18)
Group2 = c(8, 12, 16, 20, 24, 9, 13, 17, 21, 25, 10, 14, 18, 22, 26)
library(rcompanion)
mangiaficoD(x=Group1, y=Group2, verbose=TRUE, ci=TRUE, hist=TRUE)
Group  Statistic Value
1     A     Median  9.00
2     B     Median 17.00
3       Difference -8.00
4     A        MAD  7.41
5     B        MAD  7.41
6       Pooled MAD  7.41
d lower.ci upper.ci
1 -1.08    -2.81   -0.169
Group = factor(c(rep("I", length(Group1)), rep("II", length(Group2))))
Value = c(Group1, Group2)
Data = data.frame(Group, Value)
plot(Value ~ Group, data=Data)
library(coin)
median_test(Value ~ Group, data = Data)
### Asymptotic Two-Sample Brown-Mood Median Test

### Z = -2.1589, p-value = 0.03086

Addendum 2

I mentioned Vargha and Delaney’s A above as an option for effect size. This statistic is based on the proportion of observations in Group A that are greater than observations in Group B, and so on.

For the example above, the proportion of observations in Group1 > Group2 is 0.16, and the proportion of observations in Group2 > Group1 is 0.80.

These statistics are useful to report when comparing two medians.

library(rcompanion)
vda(x=Group1, y=Group2, verbose=TRUE)
Statistic Value
1 Proportion Ya > Yb  0.16
2 Proportion Ya < Yb  0.80
3    Proportion ties  0.04
VDA
0.18

Answer 2

Mood's test is really just a chi-square test in disguise (https://en.wikipedia.org/wiki/Median_test).

Hence, I recommend using the measure of effect size for chi-square tests. There are three standards used: phi, the odds ratio, and Cramer's V. The first two are only defined when comparing two samples, while Cramer's V works for nxn contingency tables.

Here are equations for phi and V:

$$\phi= \sqrt(\chi^2/n)$$ $$V=\sqrt(\chi^2/(n \min(c-1, r-1))$$

where n is the total number of observations, $\chi^2$ is the test statistic, and r/c are the dimensions of the contingency matrix (in this case it will be two x the number of groups you are comparing). If you are interested in the odds ratio, I would recommend googling it, as it is a little bit more complicated than the others.

Note I leave interpretation of the effect size to the reader. :)

https://www.real-statistics.com/chi-square-and-f-distributions/effect-size-chi-square/