# Entropy Scorer

**URL:** <https://forum.knime.com/t/entropy-scorer/86100>\
**Category:** KNIME Analytics Platform\
**Created:** [February 1, 2025, 2:05pm UTC](https://forum.knime.com/t/entropy-scorer/86100 "2025-02-01T14:05:50Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Arhan](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/a/e99b99/32.png) [@Arhan](https://forum.knime.com/u/Arhan)\
**Post date:** [February 1, 2025, 2:05pm UTC](https://forum.knime.com/t/entropy-scorer/86100/1 "2025-02-01T14:05:50Z")

</div>

Hi,

I want to find out about what the entropy is for clusters. But I’m only getting the entropy for their size rather than the amount of clusters. Any way I can fix that? I have the interval 2-10 clusters that I need to determine the entropy for

Thank you in advance

---

<div class="post-metadata">

**Author:** ![rfeigel](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/r/7ea924/32.png) [@rfeigel](https://forum.knime.com/u/rfeigel)\
**Post date:** [February 1, 2025, 4:37pm UTC](https://forum.knime.com/t/entropy-scorer/86100/2 "2025-02-01T16:37:56Z")

</div>

What clustering algorithm are you using? I assume you want to optimize the number of clusters.

---

<div class="post-metadata">

**Author:** ![Arhan](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/a/e99b99/32.png) [@Arhan](https://forum.knime.com/u/Arhan)\
**Post date:** [February 1, 2025, 5:29pm UTC](https://forum.knime.com/t/entropy-scorer/86100/3 "2025-02-01T17:29:18Z")

</div>

Sorry for any iconsistencies becuase I just started learning KNIME yesterday. But this is my workflow so far:

ARFF → Column Filter → k-Means → Category to Number → Column Filter → GroupBy → Entropy Scorer (I connected the second column filter to the entropy scorer as well as the GroupBy).

I thought the problem was that my cluster string still was showing up after I added category to number, so that is why I added an additional column filter to “remove” cluster string and only have the cluster to number, but the problem still remains,

---

<div class="post-metadata">

**Author:** ![ActionAndi](https://forum-cdn.knime.com/user_avatar/forum.knime.com/actionandi/32/76198_2.png) [@ActionAndi](https://forum.knime.com/u/ActionAndi)\
**Post date:** [February 1, 2025, 6:29pm UTC](https://forum.knime.com/t/entropy-scorer/86100/4 "2025-02-01T18:29:11Z")

</div>

Hi,  
I’m not an expert in Clustering methods. But I found this link here [Description of Entropy calculation](https://stackoverflow.com/questions/35709562/how-to-calculate-clustering-entropy-a-working-example-or-software-code) and the calculation is exact the same like in the Entropy Scorer.  
Have you checked your code?

[250201\_Entropy – KNIME Community Hub](https://hub.knime.com/actionandi/spaces/Public/250201_Entropy~xdknYtMl62MA-QtK/current-state)

---

<div class="post-metadata">

**Author:** ![Arhan](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/a/e99b99/32.png) [@Arhan](https://forum.knime.com/u/Arhan)\
**Post date:** [February 1, 2025, 7:07pm UTC](https://forum.knime.com/t/entropy-scorer/86100/5 "2025-02-01T19:07:40Z")

</div>

Thank you for the reply! I did not really understand the description, I think it was more about Python? I’m sure the calculation is the same, but I couldn’t really grasp it.

---

<div class="post-metadata">

**Author:** ![ActionAndi](https://forum-cdn.knime.com/user_avatar/forum.knime.com/actionandi/32/76198_2.png) [@ActionAndi](https://forum.knime.com/u/ActionAndi)\
**Post date:** [February 1, 2025, 7:51pm UTC](https://forum.knime.com/t/entropy-scorer/86100/6 "2025-02-01T19:51:25Z")

</div>

No the second answer gives you all the mathematics:

 ![image](https://forum-cdn.knime.com/uploads/default/original/3X/f/4/f4ac7bb117a63f91558ad1ef906bb204a38a2d99.png)

Unfortunately the Link in the Node description is broken:  
 ![image](https://forum-cdn.knime.com/uploads/default/original/3X/f/f/ff01bff98861529f1b7e96ab637749eb553ac70d.png)  
[Broken Link](https://www.uni-konstanz.de/bioml/bioml2/publications/Papers2007/WiBe07_fcum_ijar/Fuzzy%20Clustering%20in%20Parallel%20Universes_submitted.pdf)

---

<div class="post-metadata">

**Author:** ![rfeigel](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/r/7ea924/32.png) [@rfeigel](https://forum.knime.com/u/rfeigel)\
**Post date:** [February 1, 2025, 9:47pm UTC](https://forum.knime.com/t/entropy-scorer/86100/7 "2025-02-01T21:47:22Z")

</div>

You might want to think about using either the elbow or silhoutte methods. I’m on my phone right now. I’ll send you some info when i get on my computer in a few hours.

---

<div class="post-metadata">

**Author:** ![Arhan](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/a/e99b99/32.png) [@Arhan](https://forum.knime.com/u/Arhan)\
**Post date:** [February 1, 2025, 10:44pm UTC](https://forum.knime.com/t/entropy-scorer/86100/8 "2025-02-01T22:44:21Z")

</div>

Now I can see, thank you! That is exactly what I have done (or so I think). I’m getting numbers for the entropy, but those numbers are for the sizes of the clusters and not for the amount of clusters unfortunately.

---

<div class="post-metadata">

**Author:** ![Arhan](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/a/e99b99/32.png) [@Arhan](https://forum.knime.com/u/Arhan)\
**Post date:** [February 1, 2025, 11:45pm UTC](https://forum.knime.com/t/entropy-scorer/86100/9 "2025-02-01T23:45:52Z")

</div>

Thank you for the reply! Yes, I’m supposed to use the elbow method after I have gotten the entropy for all the different clusters. Only issue is, as I mentioned above, that I’m getting the entropy for the clusters’ sizes rather than the amount/number of clusters (ranging from 2 to 10).

---

<div class="post-metadata">

**Author:** ![rfeigel](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/r/7ea924/32.png) [@rfeigel](https://forum.knime.com/u/rfeigel)\
**Post date:** [February 2, 2025, 12:26am UTC](https://forum.knime.com/t/entropy-scorer/86100/10 "2025-02-02T00:26:31Z")

</div>

See if this helps. Is this a school assignment where you’re forced to us the entropy node?

> [@Clustering in Knime](https://forum.knime.com/t/clustering-in-knime/29396):
>
> Hi I have a question regarding k-means clustering.What is the best way to find the value of K ?I have heard about silhouette coefficient but I am not sure how it works. I look forward to your help. Thank you & kind regards, Nandisha

> [@Nodes for finding Best K values in clustering and Node for Significant variables](https://forum.knime.com/t/nodes-for-finding-best-k-values-in-clustering-and-node-for-significant-variables/67602):
>
> Hi , Which node is useful to Select best K value for clustering in 4.6.5 version and which node is uesd to select significant variables for building linear regression. Please help with this asap. thanks in advance .

---

<div class="post-metadata">

**Author:** ![ActionAndi](https://forum-cdn.knime.com/user_avatar/forum.knime.com/actionandi/32/76198_2.png) [@ActionAndi](https://forum.knime.com/u/ActionAndi)\
**Post date:** [February 2, 2025, 8:41am UTC](https://forum.knime.com/t/entropy-scorer/86100/11 "2025-02-02T08:41:09Z")

</div>

As far as I understood the Entropy metric it’s not possible to get an information how many clusters you need.  
It’s rather used to rate your model in a case of pre-labeled data. E.g. the entropy compares the predicted clustering against a ground truth (labeled data).  
In the case of unsupervised learning this truth is not available. There you can apply methods like “k-means” and use the elbow method to get an idea how many clusters are optimal.

---

<div class="post-metadata">

**Author:** ![Arhan](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/a/e99b99/32.png) [@Arhan](https://forum.knime.com/u/Arhan)\
**Post date:** [February 2, 2025, 10:29am UTC](https://forum.knime.com/t/entropy-scorer/86100/12 "2025-02-02T10:29:46Z")

</div>

Thank you, I will look through these and see if it’s helpful! Yes it’s a school assignment so unfortunately I’m forced to use the entropy scorer.

---

<div class="post-metadata">

**Author:** ![Arhan](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/a/e99b99/32.png) [@Arhan](https://forum.knime.com/u/Arhan)\
**Post date:** [February 2, 2025, 11:33am UTC](https://forum.knime.com/t/entropy-scorer/86100/13 "2025-02-02T11:33:12Z")

</div>

![Skärmbild 2025-02-01 183714](https://forum-cdn.knime.com/uploads/default/original/3X/9/1/91a1aec46da429fd131e6ba1e58c59b53d19f9ec.png)

Also this is my workflow and these are the numbers I’m getting

---

<div class="post-metadata">

**Author:** ![rfeigel](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/r/7ea924/32.png) [@rfeigel](https://forum.knime.com/u/rfeigel)\
**Post date:** [February 2, 2025, 2:46pm UTC](https://forum.knime.com/t/entropy-scorer/86100/14 "2025-02-02T14:46:37Z")

</div>

That’s fine but there’s nothing from the entropy node to feed to either the elbow or silhouette methods. They’re a separate issue.

---

<div class="post-metadata">

**Author:** ![Arhan](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/a/e99b99/32.png) [@Arhan](https://forum.knime.com/u/Arhan)\
**Post date:** [February 2, 2025, 3:09pm UTC](https://forum.knime.com/t/entropy-scorer/86100/15 "2025-02-02T15:09:29Z")

</div>

So if I understand you correctly my workflow is fine and those numbers the entropy scorer is showing are correct?

For the elbow method we are supposed to put together the numbers in an excel diagram and work through it from there rather than on KNIME.

---

<div class="post-metadata">

**Author:** ![rfeigel](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/r/7ea924/32.png) [@rfeigel](https://forum.knime.com/u/rfeigel)\
**Post date:** [February 2, 2025, 3:25pm UTC](https://forum.knime.com/t/entropy-scorer/86100/16 "2025-02-02T15:25:44Z")

</div>

Can you explain “put together the numbers in an excel diagram and work through it from there rather than on KNIME.”

---

<div class="post-metadata">

**Author:** ![Arhan](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/a/e99b99/32.png) [@Arhan](https://forum.knime.com/u/Arhan)\
**Post date:** [February 2, 2025, 4:09pm UTC](https://forum.knime.com/t/entropy-scorer/86100/17 "2025-02-02T16:09:33Z")

</div>

For our assignment it says we should use the elbow method to find the optimal amount of clusters and use the numbers we get for the different cluster entropies and make a diagram on excel. After we evaluate the best amount of clusters with the help of the diagram.

---

<div class="post-metadata">

**Author:** ![rfeigel](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/r/7ea924/32.png) [@rfeigel](https://forum.knime.com/u/rfeigel)\
**Post date:** [February 2, 2025, 4:33pm UTC](https://forum.knime.com/t/entropy-scorer/86100/18 "2025-02-02T16:33:17Z")

</div>

I have no idea what you mean by “Excel diagram”

---

<div class="post-metadata">

**Author:** ![badger101](https://forum-cdn.knime.com/user_avatar/forum.knime.com/badger101/32/45755_2.png) [@badger101](https://forum.knime.com/u/badger101)\
**Post date:** [February 2, 2025, 5:17pm UTC](https://forum.knime.com/t/entropy-scorer/86100/19 "2025-02-02T17:17:29Z")

</div>

![entropy](https://forum-cdn.knime.com/uploads/default/original/3X/4/3/433bdad4be56750bc4310ff6b902431648a9f067.png)

Your assignment might ask for something like this.

Having said that, entropy-based measure isn’t a good test to verify clustering numbers, it’s only good to count the total entropy after clustering is done. Elbow methods does not have to use entropy as y-value.

When I did my academic assignment years ago (which what led me to Knime, since I first discovered Knime from an academic journal), Silhouette Coefficient is enough to evaluate clustering.

Each to their own.

---

<div class="post-metadata">

**Author:** ![Arhan](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/a/e99b99/32.png) [@Arhan](https://forum.knime.com/u/Arhan)\
**Post date:** [February 2, 2025, 6:11pm UTC](https://forum.knime.com/t/entropy-scorer/86100/20 "2025-02-02T18:11:49Z")

</div>

Thank you! Yes, I think our diagram should look somewhat similar like that. I understand your point, but unfortantely I don’t have the option to choose which method. But it is definitely something I will look into.

[Next page](https://forum.knime.com/t/entropy-scorer/86100.md?page=2)
