5

I need a good reference for the methods and/or the difficulties that arise when attempting such a combination.

I've found Loughin, TM, A systematic comparison of methods for combining p-values from independent tests, Computational Statistics & Data Analysis (2004) 47(3):467–485, for example. Is there a better/more precise one?

krlmlr
  • 779
Gabriel
  • 4,282
  • Is this for multiple test correction or something else? – Bitwise Sep 27 '12 at 15:33
  • It's pretty long to explain but basically I have 3 p-values obtained through the same process (the kde.test function of R's ks package http://cran.r-project.org/web/packages/ks/ks.pdf) but applied to three different sets of 2 columns each (all 6 columns are linked because they belong to the same 'observation') – Gabriel Sep 27 '12 at 15:44
  • 1
    It sounds like you want to do a meta analysis. Fisher's combination test is one method for combining p-values. There are many books on meta analysis that could be useful to you. I will give you an answer with links to several books if you tell me I am on the right track. – Michael R. Chernick Sep 27 '12 at 20:42
  • Well I'm not a statistician so I have no idea what a meta analysis is :) (sorry) I actually do not want to perform such a combination, I just need references for the various methods of doing so and/or its difficulties (like the article I presented, to which I have no access sadly) – Gabriel Sep 27 '12 at 20:59
  • 2
    I guess I don't know why you are interested in the article. They talk in the abstract about combining p-values for a combined hypothesis. If you are combining independent data set to reach strong conclusion because of the increased sample size. That is all that us meant by a meta analysis. One way to do meta analysis (also called data synthesis) is to combined p-values from independent tests of the individual data sets. Fisher's combination test is one of many that could be tried. Apparently the paper compares 6 such methods. Meta analysis is in the list of key words in the abstract. – Michael R. Chernick Sep 28 '12 at 03:35
  • I wouldn't be surprised to find out that Fisher's test is one that they compare. – Michael R. Chernick Sep 28 '12 at 03:38
  • @MichaelChernick I mentioned the article because it mentions several methods of combining p-values. As I said, I'm not really interested in performing such a combination, rather I'm looking for sources to reference the various methods for, and/or complications of, doing so. – Gabriel Sep 28 '12 at 11:44

1 Answers1

5

One useful entry into this literature is an annotated bibliography by R D Cousins which considers most of the well-known methods for combining $p$-values and which is freely available from arXiv.

mdewey
  • 17,806
  • 1
    (+1) What is the benefit of using BibTeX on this forum? Presumably that one can copy-paste it into one's .bib file. How often do you think this is happening? I would guess that many many more people would find a hyper-link more useful; my own preference is to put a hyperlink on a short citation, e.g. Cousins, 2008, Annotated bibliography of some papers on combining significances or $p$-values or something like that. Perhaps a BibTeX citation can be provided extra in the end of an answer. I am saying this because this is not the first time you are using BibTeX and I am always wondering... – amoeba Aug 21 '16 at 21:24
  • Thank you for the article mdewey. I edited your answer to add a link since I also wondered (as @amoeba did) why you left the bibtex instead (or without) the link. – Gabriel Aug 21 '16 at 23:55
  • 1
    I would suggest to follow the advice of @amoeba and not embed BibTex for citations here. Assuming you actually do want a BibTeX citation to be available, you could cite via a Google Scholar hyperlink. If you click that link, then click Cite, then BibTex, you can get the same result. – GeoMatt22 Aug 22 '16 at 04:34
  • @amoeba (and other commenters) apologies, I had no idea I was going to perturb so many people. I have removed the formatting. – mdewey Aug 23 '16 at 09:00