2

Modified question to better explain the context of my problem:

I am studying young stars. When a star is born, it is surrounded by a disk of dust called "protoplanetary disk". Planets form in these disks, so understanding how they evolve gives information on plaent formation. Current theories and observations suggest that every star is born with one of these disks. However, different processes make these disks dissipate in about 10 million years. The usual way to study this subject is to study the fraction of stars with protoplanetary disks at different ages to see how dissipate. Past studies have found "hints" of massive stars loosing their disks earlier than low-mass stars, and therefore they may form different planetary systems. My aim is to determine the truthfulness of this dependence with stellar mass.

To study these disks, we look at the flux measured at infrared wavelengths. When you know the type of star is (lets say, you know its temperature), you can apply a stellar model. If the flux you measure is signicalty higher (defined in some way) than the expected from the stellar model (a naked star), that could mean you have additional infrared flux emited by the protoplanetary disk. Also, you need an age estimate for the star, and another one for the stellar mass if you want to compare different masses. So, there are several sources of uncertainties:

  • errors from the infrared measurements

  • errors from the estimated temperature of the star

  • errors from the age estimate

  • errors from the mass estimate.

The origin and behaviour of these uncertainties are very complicated, and usually not included in the calculations.

I have built a large sample of young stars, and I want to see which evidence there is of the stellar mass affecting the evolution/dissipation of protoplanetary disks. To do so, I have subdivided the sample in two mass and ages bins (the cuts having some physical meaning). As a result, I have four bins: "young low-mass", "young high-mass", "old young-mass", "old low-mass". Computing the % of protoplanetary disks for each of these bins is simple, but that is not enough prove or discard the mass influence. On the other hand, assigning errors to that % by error propagation is extremely complicated. Usually, one assumes simple Poisson errors, but that is not correct as it does not account for these uncertainties. That is why I thought I could use bootstrapping, and vary these quantities within reasonable ranges during the iterations to account for them.

As a result of that process, I end up with a list of % values for each bin, and therefore I can get statistical quantities from them (mean, standard deviation,…). They also provide and estimate of the correspoding PDFs.

I would like to know how to quantify the statistical evidence of these bins having different protoplanetary disk fractions, which translates into evidence of stellar mass having an impact on their evolution.

This is an example of the outcome. sample1 is "young, low-mass stars". sample2 is "young, high-mass stars". And their means and standard deviations are:

sample1: 61 +- 2

sample2: 47 +- 5

also, these are the obtained PDFs.

enter image description here

Álvaro
  • 153
  • 1
    "does it mean these two distributions are different with a 95% of confidence?" -- no. If those are actual probability densities, they chance they're different is 100%, since we can see they differ. – Glen_b May 29 '14 at 14:03
  • Hm...good point, I think I didn't ask the question properly. Edited to change the question. – Álvaro May 29 '14 at 15:13
  • What's a 'true value'? – Glen_b May 29 '14 at 16:21
  • The actual value that would be measured if no errors, biases, ... were present. I am trying to find out if the overlap area can tell me something about "how different" these PDFs are (and hence the characteristic they represent) – Álvaro May 29 '14 at 17:06
  • Alvaro, the area of overlap is an artifact of how you represent the data. If you were to smooth them much less, each of these curves would be a series of spikes--potentially with zero overlap. If you smoothed them more, the overlap would increase. The key to asking a question that is useful to you and understandable by others is to explain your objectives and ask how to address them. Asking how to fix or interpret a problematic procedure can turn into a distraction and deflect the conversation away from providing useful material for you. I urge you to edit your question accordingly. – whuber May 29 '14 at 23:23
  • What does it mean for a subsample to 'evolve'? – Glen_b May 30 '14 at 00:36
  • I have age measurements of these objects. I assume they all displayed a certain characteristic when born, and I want to see if there is evidence of different evolution of this characteristic as a function of mass, for example. – Álvaro May 30 '14 at 00:44
  • Please merge your accounts, @Alvaro, then you won't need to have your edits to your own questions approved. – gung - Reinstate Monica May 30 '14 at 23:18

2 Answers2

1

From Duncan's response, I rechecked the p-values from a two sample t-test with unequal variances. This time I created two normal distributions with the corresponding means and standard deviations and applied the algorithm. The result was not absolute zero, but almost. It seems I get p = 0 value when using the bootstraping results because the distributions obtained from it may not be perfectly normal.

Therefore, probably the correct answer is: you do not use PDFs if they are ~normal. You use the quantities that describe the data (mean, variance, ...) instead.

Álvaro
  • 153
  • Applying a t-test to bootstrapped datasets is of no use, because if there is even the very slightest difference between the means of the data, then a sufficiently large bootstrap sample will produce an arbitrarily small p-value. All you're doing is testing whether the means of the two datasets (not the populations they are sampling!) are exactly equal. – whuber May 30 '14 at 01:20
  • @whuber That makes sense, but in that case...what do you suggest? How do I get a measurement of the evidence provided by the bootstrap technique of these two samples having evolved differently? – Álvaro May 30 '14 at 06:56
  • Forget bootstrapping and focus on your problem. Two important pieces of information you have not yet revealed are (1) what exactly do you mean by "evolving"? All you have presented so far are smoothed densities of bootstraps of something; no form of evolution is apparent. (2) What kinds of differences do you wish to detect? Do you wish to see whether the means of data differ, or their variances, or something else? – whuber May 30 '14 at 14:23
  • I will updated the the question with the real problem. I didn't want to get into the details because it may get complicated, but you're right I should show the case here. – Álvaro May 30 '14 at 19:30
  • After researching through the literature, I think I could use bootstraping to estimate the mean and standard deviations of each sample. Then compare two normal distributions with these parameters using a t-test, and use the p-value to see if the null hypotesis can be ruled out. But please, correct me if I'm wrong! – Álvaro Jun 01 '14 at 00:50
0

If I'm interpreting your question correctly, here's my response:

The overlap doesn't really tell you much about the "similarities" or "differences" of the PDFs - they're either the same, or different as Glen_b says. The PDFs are really just giving you information about the associated variable in question. If the area of overlap is .05, then that means there's a 5% chance that variable A will fall in that range and a 5% chance that variable B will fall in that range. Assuming the two variables are independent, there would be a .25% chance that they would both fall in that range simultaneously.

If you're interested in comparing the two variables, then you should be able to use the PDFs to generate means and variances for your two samples and use an appropriate statistical test to compare them.

Duncan
  • 1,302
  • I edit to include the means and std of the samples, but doesn't this "overlapping coefficient" give more information? If I understand it right, the 5% overlapping area is similar to rejecting the null hypotesis with a 99.75% condifence...or not? – Álvaro May 29 '14 at 22:53
  • I would not make that interpretation of the visualization. If you want to test the null hypothesis that the two samples have different values, use a two-sample t-test with unequal variances and see what you get. – Duncan May 29 '14 at 23:57
  • That was my initial approach, using the result from each bootstraping iteration (1000 values) to do a two-sample t-test with unequal variances, but I got a p value of 0. I thought it could be due to the large number of values included and that I was doing something wrong. – Álvaro May 30 '14 at 00:08
  • You're likely setting it up wrong. If you're doing a two-sample t-test on EVERY bootstrap iteration and compiling the results, then your p is going to basically be infinitesimal (if I understand the procedure correctly). I think what you want to be doing is using the PDFs you've created above via bootstrapping, and comparing them directly (as if they were regular samples) using a two-sample t-test.

    Disclaimer: I haven't done much with bootstrapping, so I'm not positive that this is statistically sound. I'd have to investigate more to figure it out for sure.

     – Duncan May 31 '14 at 05:51