3

I have $k$ pairs of samples from different distributions. For each pair of samples I want to check if the samples are taken from a normal distribution with the same mean and variance. I assume the normal distributions for each pair is different and independent of other pairs. I do not know the mean and variance of any of the distributions.

I am doing a two sample t-test for each of the pairs. I want to combine the $k$ resulting p-values.

I could use Fisher's method but I am wondering about the distribution of the average t-value. If these were z-values rather than t-values I would use Stouffer's z-test. I could also approximate the t-distribution with a z-distribution and than use Stouffer's z-test, but I am not sure if that is a good approximation.

Is there a test like Stouffer's z-test for t-values?

Is there a better way to combine the t-values?

krlmlr
  • 779
user2224259
  • 31
  • How about converting t-values into z-values? – A. Donda Aug 23 '15 at 16:51
  • 1
    I cannot figure out what this question is asking, because it begins with the objective of checking whether samples have the same three properties--normality, means, and variances--but then proceeds to describe a procedure (a t-test) that accomplishes only one of those comparisons. After that it appears to abandon all the preceding context to focus on something else somehow related to "combining t-values." Could you edit it to clear up these uncertainties? – whuber Apr 10 '16 at 18:27
  • @whuber The OP is away for over a year and is therefore unlikely to edit. However, I think I can see what they were asking: they wanted to apply Stouffer's z-score method of combining p-values to the results from a bunch of t-tests; and they were worried that applying $z$-something-something method to $t$-tests is not a good idea because a $t$-test is not a $z$-test. So they asked for a suitable alternative. The answer to this specific confusion is that one can of course proceed with Stouffer's method. (But that other approaches make probably much more sense for the overall analysis goal.) – amoeba Apr 10 '16 at 20:36
  • Hi! Still here :) Thanks for your answers. Donda, I tried to avoid using z-values because that could be inaccurate for small sample sizes. @amoeba, that was exactly my intension. Which other approaches would you suggest? – user2224259 Apr 11 '16 at 14:52

1 Answers1

4

According to Becker's chapter on combining $p$-values in Cooper and Hedges book you can use

$$ \frac{\sum {t_f}_i (p_i)}{{\sqrt{\sum\frac{f_i}{f_i-2}}}} > z(\alpha) $$

where ${t_f}_i$ is Student's $t$ with $f_i$ the degrees of freedom $p_i$ the p-value and $\alpha$ is the desired significance value.

She does not give a reference for the method which she attributes to Winer. The resemblance to the formula for Stouffer's method is clear though.

Cooper, H and Hedges, L V
A handbook of research synthesis 1994 (Russell Sage, New York)
mdewey
  • 17,806
  • Could you edit this answer to make it readable? I am unable to fix up the formula because it is ambiguous and you haven't explained what "$t$" might mean. – whuber Apr 10 '16 at 17:14
  • Sorry about that, at the fourth attempt I realised that I cannot use double subscripts for some reason so I have removed them. Now at least it formats OK. – mdewey Apr 10 '16 at 17:36
  • Double subscripts require braces, as in $t_{i_j}$ for $t_{i_j}$. – whuber Apr 10 '16 at 18:24
  • Now sorted thanks to @whuber. I have added the citation to the book too. – mdewey Apr 11 '16 at 12:33
  • @mdewey, thanks for your answer! Isn't the suggested method equivalent to Stouffer's z-test? – user2224259 Apr 11 '16 at 14:58
  • As all $f_i$ tend to infinity, yes, it is equivalent to Stouffer's method. – mdewey Apr 11 '16 at 15:30