Bug: Data table filtering functionality shows removed categories

Hello to the team,
having used the filtering functionality of the node output data table view quite a lot since switching to 5.1, I would like to report on an issue I have with it. Quite often, the filter displays values that are no longer in the data, having been removed in preceding nodes. Or in other words: after one or more nodes change (reduce) the data, the filter still shows values that do not exist in the data anymore.
This is easily reproduceable (see attached screenshots).
KNIME_Screen_1
KNIME_Screen_2
Best regards,
Jan

Hi Jan,
This is actually a “feature” and has been in KNIME for a long time. The filter nodes leave the column spec intact and with it also the possible values. You can add a Domain Calculator after the filter to recalculate the possible values.
Kind regards,
Alexander

Hello Alexander,
thank you for the quick feedback!
For someone (like me) who came from Excel to KNIME, and needs to do a lot of small analysis steps on data, this is a.) really confusing and, b.) now that I know how it works, against my needs as an analyst.
I understand this probably has a deep background/history, but it does not encourage people to let go of Excel (I am in the process of trying to get some more people in out company to use KNIME).
Best regards,
Jan

PS: To illustrate why this bothers me: I would like to use the filtering functionality to check whether certain processing steps have performed as intended, and what my data looks like after them, and for this kind of integrity cross-checking, the filtering funtionality cannot be used.

1 Like

Hi Jan,
I agree that it is counterintuitive and should probably be changed. This has come up quite often in the past. However, I think the problem is that just changing it now could break existing workflows. So the filter nodes would need to be deprecated, which is a big step. However, I’ll ask internally if we could consider it. I am 100% on your side on this :smiley:
Alexander

3 Likes

Hey,
Just a heads-up. The Row Filter nodes are due for a rewrite and our developers will keep in mind the domain calculation issue for that. There might be an option to have one or the other behavior.
Kind regards,
Alexander

3 Likes

As I finished writing this, I think @AlexanderFillbrunn has already beaten me to it if I’m reading the last post correctly. :slight_smile:

… whilst I agree that such changes to nodes can have implications on existing workflows, rather than deprecating existing nodes, what could be done (still not entirely trivial, but a possibility!) is what I do with components where I wish to change the way they operate - introduce a checkbox or other option in the config that enables new behaviour whilst defaulting to old behaviour.

In this case maybe a “recalculate domain after filtering” checkbox, where the default is the existing behaviour (i.e. don’t recalc), could be introduced. That way it gives the possibility of more intuitive results without breaking existing workflows. Just a thought.

3 Likes