4

So my data consists of a distance matrix for some points, and a table classifying the points as either red or green. I want to know if there is any difference between the red and green points. I did UPGMA clustering and they don't look different (doesn't seem to be any tendency for red or green to cluster together) but I don't know how to express the significance of this.

I guess the question would be something like "what is the chance of getting a distance matrix like this, if the points were all randomly distributed".

Johnny Brown
  • 141

5 Answers5

3

Here is a simple solution that should give you a taste of how to solve the problem. Whether this solution is satisfactory would depend on your actual application.

If you want to determine how usual or unusual your distance matrix is compared to other arrangements of the points then you can use the following approach. You will need a measure of how well any set of red and green points are clustered. A very simple metric would be as follows. 

Initialise the metric to zero.
Foreach data point
    If the nearest neighbour of a point is in a different group (ie different colour) then add this distance to the metric.

Now, you can label the data points randomly with red or green and work out the metric for a random labelling. Repeat this random labelling many times and record the metric each time, allowing you to determine distribution statistics for the metric.

You can also calculate the metric for your actual set of data points and compare this to the distribution of metric data.

Some things to note. If you have two tightly clustered groups that are well separated then the metric would be zero....ie all neighbours belong to the same group.

The metric can be as simple as I described or more complex, depending on the application of the results. This sort of random reladelling is common. If you have a relatively small number of points you could work out the metric for all possible permutations.

You can find more on this topic here http://en.wikipedia.org/wiki/Resampling_(statistics)

martino
  • 1,690
  • I second this answer. A variant: Randomly permute your red and green labels under the null hypothesis that there is no difference between them, form 2 clusters, and calculate the average silhouette value (http://en.wikipedia.org/wiki/Silhouette_%28clustering%29). Repeat this 10k times and compute the fraction of average silhouette values that are at least as large as the average silhouette value of your actual data -- this is the probability of observing a dataset at least as extreme as your dataset, given that the null hypothesis is true. – lockedoff Feb 16 '12 at 17:05
2

You could use point-biserial correlation (and its significance) between variable "distance" and binary variable "points are different colour (1) vs both points are same colour (1)". This will give you numerical impression of whether points of the same colour tend to cluster together according to the distances.

ttnphns
  • 57,480
  • 49
  • 284
  • 501
1

For each object, sort the other objects by their distance, compute the ROC curve with respect to the objects class and the area under this curve. If the distance matrix is helpful, the value should be significantly closer to 1.0 than to 0.5

Has QUIT--Anony-Mousse
  • 42,358
0

Use a permutation test:

  1. Calculate your distance metric
  2. Randomly shuffle the red/green status of the points
  3. Recalculate the distance metric on the shuffled points
  4. Repeate steps 2 and 3 a bunch of times (1999 or 9999 would be good)
  5. Compare the metric for the original data to the metrics from the permuted metrics.
mpiktas
  • 35,099
Greg Snow
  • 51,722
0

Another option is adonis, in the R package vegan.

From the docs:

adonis is a function for the analysis and partitioning sums of squares using semimetric and metric distance matrices. Insofar as it partitions sums of squares of a multivariate data set, it is directly analogous to MANOVA (multivariate analysis of variance). M.J. Anderson (McArdle and Anderson 2001, Anderson 2001) refers to the method as “permutational manova” (formerly “nonparametric manova”). Further, as its inputs are linear predictors, and a response matrix of an arbitrary number of columns (2 to millions), it is a robust alternative to both parametric MANOVA and to ordination methods for describing how variation is attributed to different experimental treatments or uncontrolled covariates. It is also analogous to redundancy analysis (Legendre and Anderson 1999).

Johnny Brown
  • 141