1

I want to test for statistical significance of the difference between two independent samples of normally distributed measurement. However instead of comparing means or overall distributions I want the test to focus on the right tail (say 75th percentile and above) of the distribution more than the "middle" part or the left tail.

Typical tests like Welch's t-test or Mann-Whitney U-test treat all distribution regions equally.

One possible solution that came to my mind was to filter both samples by removing measurements where the value is below the 75th percentile (applied independently to each of the two samples).

An answer to a somewhat similar question mentioned the Kolmogorov-Smirnov test (https://stats.stackexchange.com/a/568980/405082); however it is unclear how to apply it to the specific percentile (maybe using the filtering I described above?).

My question is: can I take a subsample by percentile of original samples in order to estimate statistical significance of the right tail difference?

I am using Python, if that matters.

considerate_shrimp
  • 41
  • 1
    Why subsample? That loses information. Use a standard test, such as a permutation test, a nonparametric rank-based test, a bootstrap (if you have enough data), or a Normal-theory test based either on the estimated variance of the percentile or its estimated distribution. – whuber Jan 18 '24 at 15:43
  • @whuber permutation test with quantile as a target statistic looks promising, I will try it out.

    As for rank-based test, can you provide some details? I am lost on how to apply something like Mann-Whitney U-test to a quantile.

     – considerate_shrimp Jan 22 '24 at 09:11
  • Adapt the ideas explained at https://stats.stackexchange.com/a/284970/919. – whuber Jan 22 '24 at 13:44

0 Answers0