What does the size of the Wilcoxon rank sum test 'W statistic' indicate?

Question

I wish to see whether the average meal masses of two species are significantly different, and I have sample sizes of >800 for both. My Wilcoxon rank sum test returns a W = 330520 value - how should I interpret such a high value?

Answer 1

If you want to better understand what this value means, you inevitably need to understand how this value is computed.

TL;DR

Wilcoxon rank-sum test computes number of events $X > Y$

nSamples <- 100
samplesGroup1 <- rnorm(nSamples, mean = 2)
samplesGroup2 <- rnorm(nSamples, mean = -1, sd = 3)

theW <- sum(outer(samplesGroup1, samplesGroup2, '>'))
wilcox.test(samplesGroup1, samplesGroup2)$statistic == theW # = TRUE

For a large sample size, e.g. $800$, the maximal value of the statistic is $800^2$. If $W=330520$, it means that $330520/800^2$ of 'greater' comparisons is true. That is, $P(X>Y)$ ~ 50% and the two distributions are kind of indistinguishable on ordinal scale.

Longer version

Wilcoxon rank-sum test is used on samples to compare whether their distribution differ. Please make sure that your experiment, according to the assumptions, contains independent samples.

E.g. say we have following data:

library(tidyverse)

nSamples <- 100
samplesGroup1 <- rnorm(nSamples, mean = 2)
samplesGroup2 <- rnorm(nSamples, mean = -1, sd = 3)

sampleDF <- bind_rows(
  data_frame(group = 'gr1', value = samplesGroup1),
  data_frame(group = 'gr2', value = samplesGroup2)
)

To compute the statistic we need to assure assumption 2: the responses are ordinal. Therefore we transform all values of samples to ordinal scale (rank), not per each group but as a whole!

rankedDF <- sampleDF %>%
  mutate(rankedValue = rank(value)) %>%
  arrange(rankedValue) %>% # this is important for plotting
  select(- value) %>%
  group_by(group) %>%
  mutate(id = 1:n()) %>%
  spread(key = 'group', value = 'rankedValue')

Wilcoxon static computes number of events where values from one group are greater than values from another group. That involves comparing every value of one group to every values of another group. (We can use R's outer function to do exactly this).

 outer(samplesGroup1, samplesGroup2, '>')

will yield a matrix (number of samples in group 1 x number of samples in group 2) of TRUE and FALSE, where TRUE indicates that value in group 1 is greater than another value in group 2.

Visually it would look like this for 100 samples per group:

expand.grid(g1 = 1:nrow(rankedDF), 
            g2 = 1:nrow(rankedDF)) %>%
  # mutate(greater = rankedDF$gr1[g1] > rankedDF$gr2[g2]) %>%
  mutate(greater = as.vector(outer(rankedDF$gr1,
                                   rankedDF$gr2, ">"))) %>%
  ggplot(aes(x = g1, y = g2, fill = greater)) +
  geom_tile(color = "black") + 
  theme_bw()

Now if you count the TRUEs, i.e. sum(outer(samplesGroup1, samplesGroup2, '>')), this will be the W-statistic.

That should answer your question: a high number is due to the large sample size of >800.

To dig a little deeper, how can you interpret this number? Well, if you heard about the area under the curve, that is exactly what we see and can compute from the W-stastistic by dividing by the number of comparisons (i.e. number of squares in the plot).

Answer 2

You might consider two statistics that can be derived from the Wilcoxon rank sum test and will be easily interpretable by your audience.

The first is the p-value.

The second, as @Glen_b mentions in the comments to the question, is the probability that an observation in one group is likely to be larger than an observation from the other group. This is often called Vargha and Delaney's A, and they provide interpretations as to whether one might consider these probabilities "small", "medium", or "large", though of course these interpretations aren't universal across disciplines and specific cirumstances. VDA is closely, and linearly, related to Cliff's delta. †

If is fine, of course, to also report your sample size and W value, but your audience will be more familiar with p-value and VDA. (Although be sure to explain what VDA means, because it's not that common).

---

† Edit: A few additional comments on VDA. Since VDA is the probability of an observation in one group being greater than an observation in the other group, a VDA of 0.50 represents no effect. VDA values closer to 1 correspond to a high probability that observations in one group are greater than the other, while values closer to 0 correspond to observations in the the other group being greater.

Assuming that tied values are accounted for properly:

1) VDA (A, B) + VDA (B, A) = 1

2) Cliff's delta, or dominance statistic, is delta = VDA(A, B) - VDA(B, A). It ranges from -1 to 1, with 0 being no effect.

3) Odds ratio = VDA (A, B) / VDA (B, A). (Or often, its inverse; usually whichever is larger.)

(See, Grisson and Kim, 2012, Effect Sizes for Research, 2nd, Chapter 5.)