1

I am PhD Biologist. I have 5 populations of infected moth larvae (3 inbred, and 2 outbred). The infection in this case is spread vertically (from mother or father to offspring). Normally the virus is spread horizontally.

Analyses through GLM, GLMMs and (arcsine transformed) LMMs reveals that there is no significant difference between these populations. (Note: as block and day larvae were analysed for infection through PCR where ordered events, it could be argued that they are fixed or random effect hence GLM and GLMM analysis. Arcsine tranfomed LMMs are occasionally used for proportional data when events or very rare or very common.)

I suspect that the generally low infection rate may mean that although the number of larval hosts is high, the number of infected hosts is low, leading to a small sample size. Hence the non-significant result. Would it be the case that if the experiment were repeated, an unfeasably large sample size would be needed to show any difference between the populations? Therefore, would looking at a similar moth virus, with a greater rate of vertical transmission be a better idea, than pursuing it in this system?

I want to do a power analyses of the to verify this, for the write up. I understand that power analyses is done in the planning of experiments. Could it give you an idea of whether an unfeasably large sample size is needed to show an effect? I have had no experience of doing this, so if anybody could point me to books or webpages that could help I would be grateful.

The data is available under the name fulldata here.

gung - Reinstate Monica
  • 145,122
Martin David Grunnill
  • 53
  • Sorry for the few typos in that. I really should proof read a post. Monday mornings. – Martin David Grunnill Oct 27 '14 at 11:48
  • 1
    Power calculations are for planning a new study, not for analyzing existing data. Your nonsignificant result is due to the fact that you don't have enough data to conclude a difference of the size you observed. That's the end of the story, you can't milk it for more with a power analysis. – Russ Lenth Oct 27 '14 at 15:13
  • I have rephrased what I am asking. As from looking from your comment that I have phrased this badly. I hope this clarifies what I mean. – Martin David Grunnill Oct 27 '14 at 16:28
  • 3
    Yes, having a very rare outcome in a study causes power to decrease considerably. There is a good deal of information on how to simulate power analyses in a closely analogous situation at the linked thread. If you still have a question after reading it, come back here & edit your Q to state what you've learned & what you still need to know. That way we can provide the information you need rather than just duplicating information that already exists elsewhere & didn't help you enough. – gung - Reinstate Monica Oct 27 '14 at 16:47

1 Answers1

2

I'm concerned that the right answer to the wrong question might be worse than none at all. There is a tendency in statistical practice towards "asterisk blindness" -- focusing on statistical significance and forgetting that you are trying to do science. The goal of statistics is not to confirm what you wish were true, it's to find out what really is true.

So the first thing I wonder about is whether or not the differences you observed are important or meaningful. If not, then you didn't find any differences, and that is as it should be. If not, then you don't have enough data, in which case you need to plan for a future study that is adequately powered based on what would be a meaningful difference (not your estimated differences) between two groups.

Again, what is missing here is the scientific context. You can't use statistics to decide what is important scientifically; you have to use scientific opinion.

Russ Lenth
  • 20,271
  • I get what you mean. You run the risk of getting so lost in statistics and the search for significance, that it detracts from being a sensible scientist. – Martin David Grunnill Nov 05 '14 at 10:13