Comparing clustering of sequences in datasets with different N?

Question

When doing sequence analysis using a package such as TraMineR, one can calculate a clustering based on Optimal Matching (OM) distances, and then plot it as a tree. I use agnes to do it, roughly like this:

sequences.sts <- seqdef(sequences.sts)
ccost <- seqsubm(sequences.sts, method = "CONSTANT", cval = 2, with.missing=TRUE)
sequences.OM <- seqdist(sequences.sts, method = "OM", sm = ccost, with.missing=TRUE)
clusterward <- agnes(sequences.OM, diss = TRUE, method = "ward")
plot(clusterward, which.plots = 2)

This gives me a plot of the cluster diagram, and it also gives me an agglomerative coefficient. However, ?agnes.object notes that the agglomerative coefficient (ac) grows as the dataset grows, and therefore it is unsuitable as a way of comparing datasets of different size.

Is there any other way of comparing the overall "degree of clustering", or overall "degree of alignment" in a sequence dataset that allows us to reliably compare datasets of different sizes?

score 3 · Answer 1 · edited Apr 13 '17 at 12:44

One usual way to compare the degree of clustering independently from the dataset size is taking the mean of the silhouette.

E.g. in MATLAB you can see the dataset size won't change the mean of the silhouette

rng('default');  % For reproducibility

for n = 1:100:1000
    % Not clustered
    X = [randn(n,2);randn(n,2)];
    cidx = kmeans(X,2,'distance','cityblock');
    s = mean(silhouette(X,cidx,'cityblock'))

    % So so clustered
    X = [randn(n,2)+ones(n,2);randn(n,2)-ones(n,2)];
    cidx = kmeans(X,2,'distance','cityblock');
    s = mean(silhouette(X,cidx,'cityblock'))

    % Clearly clustered
    X = [randn(n,2)+5*ones(n,2);randn(n,2)-5*ones(n,2)];
    cidx = kmeans(X,2,'distance','cityblock');
    s = mean(silhouette(X,cidx,'cityblock'))
end

(FYI: interpretation of silhouette value).

More generally, most internal indices can be normalized by the dataset size, thereby allowing to compare the degree of clustering independently from the dataset size.

score 3 · Accepted Answer · answered Nov 25 '13 at 18:02

3

As said, in the other answer, cluster quality indices can be used for this purpose. More generally, these indices can be used to compare clustering of different number of groups or obtained with different clustering algorithm. In R, these indices are available in the WeightedCluster library. For more informations see here

http://mephisto.unige.ch/weightedcluster/

What do you want to infer when comparing clustering of dataset of different sizes?

answered Nov 25 '13 at 18:02

Matthias Studer

1,018

1

I simply want to understand the average distance between sequences in one dataset, and compare that with another - so that I can say that "the sequences in dataset A are structurally more similar to each other than the sequences within dataset B are to each other". – histelheim Dec 06 '13 at 22:55
1

In this case, you should look at "Homogeneity of discrepancy" test. It tests whether the averages distances are significantly different accross groups. Look at the Levene test provided by dissassoc in TraMineR. The test is decribed here: Studer, M., Ritschard, G., Gabadinho, A. & Müller, N.S. (2011), "Discrepancy Analysis of State Sequences", Sociological Methods and Research. Vol. 40(3), pp. 471-510. – Matthias Studer Dec 07 '13 at 10:38

Comparing clustering of sequences in datasets with different N?

2 Answers2