0

I'm using Mclust to perform a Latent Profile Analysis. I want to observe the entropy values for different cluster solutions. I have used the mclust addons function EntropyGMM, but I find values above 1, which I also see in the paper about entropy in mixture modeling from the authors of mclust (https://doi.org/10.1016/j.csda.2022.107582). However, as I understand it from other papers that used entropy to evaluate cluster solutions, entropy should be between 0 and 1.

Why are entropy values in mclust not ranged between 0 and 1? How should I interpret these higher values and/or should I transform these values in some way to find values between 0 and 1?

Nick Cox
  • 56,404
  • 8
  • 127
  • 185
nickydv
  • 1
  • Thank you, this is definitely helpful. If I understand it correctly, the entropy value that I get, can be scaled by log k (where k is the number of clusters). I'm not yet sure how to implement that into R, although that might also be due to a mathematical language barrier (never learned maths in English). I'm not sure what it means to "scale by" log k (what is the specific mathematical operation I should use?). – nickydv Oct 06 '23 at 08:39
  • I'm fairly ignorant in this, but it seems to me that implementing mclust through the tidyLPA package gives you the relative/normalized entropy value (that ranges from 0 to 1). – Sointu Oct 06 '23 at 08:42
  • I am not familiar with mclust, latent profile analysis or R but this question does seem to hinge on relative entropy being needed not entropy. The difference is just a division, but make sure the base of logarithms is the same in each calculation. – Nick Cox Oct 06 '23 at 09:24

0 Answers0