0

I am new to this and would like to confirm my hypothesis testing.

I have right skewed data with no ties in the ranking. N1= 56765 and N2 = 56895, U = 1,583,736,025. Now I am trying to find whether to reject the Null hypothesis or not. I calculated the critical value of U using the formula mentioned in the image with z=1.960 and critical value of U is 1,625,662,818. Does this mean that the two population are not equal? enter image description here

enter image description here

enter image description here

enter image description here

Sumit Saini
  • 1
  • 3
    Usually, hypothesis testing with such a large dataset accomplishes nothing. What is the statistical question you need to answer? It must be an important one to merit analysis of so many data--but with so many data, surely you can learn far more than that there's some kind of difference between two groups! – whuber Mar 31 '22 at 02:36
  • The variable is life of a hardware component in the machine. We have made changes on how we calculate the life. So the statistical question I am trying to answer that new calculation will have significant change in the life of that hardware component among the population. N1 have life value with old calculation and N2 have life value with new calculation. Once I have determined the statistical significance, I will have to determine the impact on the life with new calculation. – Sumit Saini Mar 31 '22 at 03:31
  • 4
    Statistical significance seems of little practical relevance. You have an old calculation; you have a new one: report the mean difference and a standard error for it. – whuber Mar 31 '22 at 03:49
  • Difference in new life value vs old life value is right skewed distribution, will it be relevant to report mean difference? I was thinking to report median difference. Standard error is a good measure for skewed distributions? I have added the histogram of difference in new life value and old life value above with the question. – Sumit Saini Mar 31 '22 at 06:09
  • It depends on how skewed the distribution is: see https://stats.stackexchange.com/questions/69898. It has to be very strongly skewed to lead one to suspect the standard error of the mean would be misleading with groups as large as yours. One reason for focusing on the mean, though, is that this seems likely to be more relevant in your application than the median: the mean is directly related to total total lifetime whereas the median is not. – whuber Mar 31 '22 at 13:25
  • Got it. To make sure I understood correctly. Can you please review the sample data image and calculation I have added to the original question. – Sumit Saini Mar 31 '22 at 16:25
  • 1
    You have paired data. Analyze the differences (or ratios) of the two calculations. This will be much more powerful than analyzing the groups separately. – whuber Mar 31 '22 at 16:31
  • The x-axis in the histogram image above is the differences in new vs old life value. I am not sure what insight to report from that histogram. – Sumit Saini Mar 31 '22 at 16:37
  • 1
    The new values are consistently larger than the old ones. You would get more insight by looking at the ratios, but that consistency will remain. – whuber Mar 31 '22 at 16:55

1 Answers1

1

Comment: As @whuber has commented, this is not a situation in which hypothesis testing is likely to be useful. Consider the fictitious data sampled in R (from right-skewed gamma populations) and summarized below:

set.seed(401)
x1 = rgamma(57000, 4, 1/400)
summary(x1);  sd(x1)   
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  85.09 1016.90 1466.74 1598.92 2045.88 9062.14 
[1] 799.5738
x2 = rgamma(57000, 4, 1/399)  
summary(x2);  sd(x2) 
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  53.02 1014.25 1463.89 1592.21 2032.31 7999.17 
[1] 794.2221

For these particular data, the second sample has slightly smaller max, min, quartiles, median, mean and SD. Without knowing the context of your experiment, it is not possible to say whether these differences are of practical importance. Also, your real data might show other differences.

Boxplots show some of the slight differences noted above and happen to show a slightly different pattern of high outliers between the two samples.

boxplot(x1, x2, horizontal=T, col="skyblue2")

enter image description here

The Wilcoxon rank sum test is equivalent to the Mann-Whitney U test, but uses a different test statistic. As for your data, the test does not show a statistically significant difference at the 5% level.

wilcox.test(x1, x2)

    Wilcoxon rank sum test 
    with continuity correction




data:  x1 and x2
W = 1.631e+09, p-value = 0.2448
alternative hypothesis: 
 true location shift is not equal to 0

With such large sample sizes, it makes better sense to look at the data descriptions than at the results of the test. The test looks only for a difference in location and descriptive statistics may suggest other differences-- some of which might be of practical importance.

BruceET
  • 56,185
  • 1
    I like the idea of box plot comparison. The context of the experiment is for each machine there is a mathematical formula to calculate life of a hardware component. Now I am making changes to the formula and want to understand the impact of the new calculation on the life of the hardware component statistically. I have added sample data image to the original question. – Sumit Saini Mar 31 '22 at 16:28
  • "The Wilcoxon signed rank test is equivalent to the Mann-Whitney U test" -- this seems a little misleading as a bare statement given that one is for a paired test and one is not. – Glen_b Apr 01 '22 at 00:13
  • 1
    @Glen_b, typo corrected. Rank sum test was intended and illustrated just below. Thanks. – BruceET Apr 01 '22 at 06:27