1

This question is related but I want to check if my idea is valid.

I have two peaks and I want to check if they are the same. They are the linecut of an image. The background noise in the image is not Gaussian. I previously tried to compare the difference but I don't think it is a promising route.

What I am considering is carrying out the test on the two peaks as though they were each a distribution already. I am not going to bin it as this would lose information. In effect they are already binned as each pixel in the image corresponds to the amount of light that was incident on it. This is why I suspect that I can treat them as two distributions.

The crux of my question is if it is justifiable to treat each peak as a distribution and then to analyse these peaks with methods like the KS test.

Here is an image of my peaks. enter image description here

zaphod
  • 13
  • @Glen_b Thank you for your response. Is there a separate check I could do to compare the similarity of the shape of the two curves? – zaphod Jul 11 '23 at 02:31
  • Welcome to Cross Validated! What randomness do you have? There isn’t really statistical testing without randomness. – Dave Jul 11 '23 at 02:37
  • @Dave This data comes from an image taken with a camera. The setup isn't perfect so it has introduced some (non-Gaussian) noise in the image. I believe that the two peaks should be the same but I don't have a way of quantifying their similarity. All of the goodness of fit tests seem to be intended for distributions and not curves. – zaphod Jul 11 '23 at 02:49
  • What is the red vs the black? // 2) What exactly do you want to know? You can see that the curves differ. If you want to quantify the difference, that is not a matter of statistical hypothesis testing (which is fine, but then you’re going down a rabbit hole by pursuing statistical testing). // 3) Functional data analysis is an approach to treating curves as individual observations (at least philosophically) and might be worth considering.
    • – Dave Jul 11 '23 at 02:56
  • I took two images. The red is from image 1. The black is from image 2. I would like to attain a p-value that allows me to accept or reject the hypothesis that these two peaks are the same. – zaphod Jul 11 '23 at 02:58
  • You see that they’re not the same. What do you want to know beyond that? // If you took two images, how do you wind up with four graphs? – Dave Jul 11 '23 at 03:00
  • I cannot say that they're not the same when the background noise seems to be on the same scale as the difference between the two peaks. I took two images in four separate runs. – zaphod Jul 11 '23 at 03:02
  • But what do you want to know? It sounds like you’re asking if you photographed the same object. You’re the photographer: did you? What is there beyond that answer? – Dave Jul 11 '23 at 03:13
  • This image is not from a camera per se. It's the phase shift recovered from a hologram. Each image comes from analysing a different reference beam. If my setup is well-aligned then these two images are the same. If my setup is poorly aligned then they are not the same. I want to be able to quantify how well-aligned my setup is. In your photography analogy, I took two images with two separate cameras and I want to know if there is a difference in the two images I got as this would imply that there is an issue with one or both of the cameras. – zaphod Jul 11 '23 at 03:18
  • 1
    So would it be fair to say that one image is the result of some “true” curve, on top of which is some noise each time you take a photo, and then the other image also have a “true” curve that is perturbed in each photograph by some (possibly different) noise, and you want to quantify how close those two “true” curves are even though you only observe them along with the noise that you cannot remove? (That would be a statistics problem, quite analogous to testing if two means are equal.) – Dave Jul 11 '23 at 03:28
  • Yes, you've got it exactly right. Apologies if that was not obvious. – zaphod Jul 11 '23 at 03:30
  • Are these "peaks" from a Fast Fourier Transform of a signal? – AdamO Jul 11 '23 at 04:40
  • @AdamO An FFT is involved in generating them, yes. – zaphod Jul 11 '23 at 04:52
  • I’m glad we managed to resolve your issue. I encourage you to look into FDA and ask questions about it on here. Where you may find frustration is that few people do it (I’ve found that frustrating). I’ve also struggled to come up with measures of effect sizes. Sure, functional ANOVA can reject when sample sizes are large like I’ve had when I’ve wanted to run FDA, but how different are they? I can quantify a difference in means by subtracting the means and looking at the resulting number. This is not so straightforward for entire curves. // @AdamO What impact do you think that would have? – Dave Jul 11 '23 at 11:02
  • @Dave, thank you for helping me. I did some further reading and the method of block bootstrapping seems to be promising. My sample does not appear to be exchangeable as there is some correlation between adjacent data points so it is similar to a time series in this respect. I would use it to see if there is a significant difference between the means. – zaphod Jul 11 '23 at 12:46
  • 1
    I have trouble imagining how an assessment of means is at all helpful for your overall task. I am not even sure about what mean you would calculate. FDA seems like the way to go to analyze entire curves like you want to do. – Dave Jul 11 '23 at 13:01