Issue in evaluating the performance of my "clustering algorithm" using NMI, ARI when the "ground truth" is available?

Question

(**Edited the question after the initial comments)

Suppose,

Ground_truth_data = [1, 1, 1, 1, 1, 1, 1];

Clustering_result = [1, 1, 1, 1, 1, 1, 2];

Here, as you can see, there are "7" instances of data and two classes with labels "1" and "2".

Q1) Looking at this result, what can be said about the performance of my clustering algorithm that produces the "Clustering_result"?

Here's what 'two' of the widely used (have seen in many reseach papers) external clustering measures says (https://scikit-learn.org/stable/modules/clustering.html#clustering-performance-evaluation):

ARI = metrics.adjusted_rand_score(Ground_truth_data, Clustering_result)
ARI = 0.0 #Adjusted Rand Index Result

AMI = metrics.adjusted_mutual_info_score(Ground_truth_data, Clustering_result)
AMI = -1.4018874593092454e-15 #Adjusted Normalized Mutual Information

Q2) Does these results mean my clustering algorithm is performing badly? According to me, NO!! Since out of "7" instances "6" are clustered correctly. (Please correct me if I am wrong)

Q3) Why ARI, NMI result in values close to ZERO in the above case?

Q4) In my dataset of 300 different types of data, the maximum value possible for the no. of clusters is "8". Also, in this dataset, there are many instances where a situation similar to the above example have been noticed. In such a case, what are the suitable performance measures (that are consistent) to evaluate the performance of my clustering algorithm?

Note in particular alto's answer in the proposed duplicate, especially this page alto links to. — Stephan Kolassa, Apr 12 '19 at 10:18
I have reframed the whole question. But why is it still marked as duplicate?? — Ultra_Champ, Apr 22 '19 at 10:36
Q1&2: we can't say whether this performance is good or bad without knowing your context. A given performance may be very bad if the task was easy, or very good if the task was hard. How to know that your machine learning problem is hopeless? Q3&4: I can't say whether these warrant reopening your question. Have you looked at Anony-Mousse's answer? It sounds like Q3 should be standard and be answerable with basic knowledge of clustering. You may improve your chances for reopening if you indicate clearly why standard textbooks do not help. — Stephan Kolassa, Apr 22 '19 at 15:19
Q2: it is bad because it's close to random guessing. Q3 has been discussed here before, too. These methods are not defines for constant results. Please search for existing questions and answers! — Has QUIT--Anony-Mousse, Apr 22 '19 at 15:32

score 1 · Answer 1 · answered Apr 12 '19 at 14:17

1

Please see a text book on this subject.

It's so widely known (ARI, NMI, etc.) that it's even discussed in Wikipedia...

Also use the search function, e.g., Evaluation measures of goodness or validity of clustering (without having truth labels)

While that question at first sight is on unlabeled evaluation, reading the answers will guide you to the extrinsic evaluation, too.

answered Apr 12 '19 at 14:17

Has QUIT--Anony-Mousse

42,358

Do the OP's edits warrant reopening the question? – Stephan Kolassa Apr 22 '19 at 15:19
At least some of these new questions are also already answered on SE/SO. He really needs to study the literature and related questions here. – Has QUIT--Anony-Mousse Apr 22 '19 at 15:31

Issue in evaluating the performance of my "clustering algorithm" using NMI, ARI when the "ground truth" is available?

1 Answers1