Inconsistent Row Filter node behaviour with Row Number filter

[takbb]

During the creation of a workflow to assist with a question on the forum, I ran into strange and inconsistent behaviour on the Row Filter node with respect to row numbering.

In searching I now see that this was mentioned back in 2017, but haven’t seen it actually documented or acknowledged as a bug, which I believe it is.

Row filter mystery

Hi, The answer must be stupid but... I've got a table with 5 lines. I want to select the last row. So I use the row filter and use the include row by number with the First row number = 5 and the Last row number = 5.Ok it works. Now I want to do it dynamically to select only the last row whatever the table dimension. Well, I use the Extract Table dimension node followed by the Table Column to Variable and adjust the flow variables panel to RowRangeStart=Number Rows and Row RangeEnd=Number Rows…

Whilst additional comments in that 2017 post gave background regarding 0 or 1 based numbering schemes (which are now well documented as inconsistent across nodes) they did not actually deal with the fundamental issue that in this case that there is an inconsistency within the same node, causing undesirable changes in behaviour when using a flow variables.

By way of demonstration to save a thousand words, I have here a workflow containing a Table:

Inconsistent Row Filter numbering.knwf (14.6 KB)

If I manually filter, to return rows by number (rows 1 to 3)

I get the expected output:

The config dialog is quite clearly 1-based. It does not allow an entry of 0.

HOWEVER…
If I pass 1 and 3 as flows variables, as a proxy to entering it manually, then I expect (as with ALL other nodes that I have used), the values 1 and 3 to be respected as if they had been entered manually…

The Row Filter returns the following:

Only by passing the values 0 and 2 in flow variables as my start and end rows does it return the required result.

Of course I can see that for backward compatibility, it is possibly not desirable to fix this behaviour, but I thought it should be documented. Whilst I fully accept that there are differences in the way nodes approach row numbering, this issue isn’t actually about that. This is about the fact that passing a value as a flow variable as a proxy to entering that SAME value manually actually changes the result of the node, and this is definitely not ideal.

[ipazin]

Hello @takbb,

this is known and there are many topics around it so it’s kind of documented. One can also use Row Filter (Labs) node which addressed this confusion.

Off-by-one error in Row Filter node when using row ranges with variable override

I think, there is a problem with the row filter when using variable overrides to specify a row range. Here is a short example: [image] This is a simple little table and I filter rows two through to four. The result is as expected*. Now, I simply override the range limits in the row filter with some variables, which have the same values, two and four respectively. But now I get: [image] Please note the subtle difference. This happens on KNIME 4.0.2. To be honest: I’d rather have here a lef…

And as you can see in above topic there is ticket to document this within node description. Will add +1 there.

Br,
Ivan

[takbb]

Thanks @ipazin, I did a search but only found the article listed, so apologies for not having discovered that already before posting. I think having found the 2017 article and seeing it closed with no mention of it being a bug, I decided to investigate further.

But anyway, thank for the +1 on adding to the node documentation (which ironically I did actually look in before posting to see if it was mentioned there), but at least now it’s even better documented , and anybody finding the article that I found will now find links here and then a link to the bug!

Incidentally is there any publicly available list of known issues that is searchable on a node-by-node basis? I know that whilst generally on searching the forum, I mostly find what I’m looking for, sometimes even for something I know I’ve posted I don’t always find it right away because I didn’t quite get the right keyword.

Thanks for the tip on the presence of Row Filter (Labs) node. I hadn’t really noticed that before so it might prove useful in future. Having just looked at the scary (sorry, I mean “advanced and sophisticated”) config for it I think for this particular use case I’ll stick to just dealing with “subtracting 1” in the flow variable… now I know

For completeness though, after a little playing around with the Labs node (and some strong coffee!) here is the “equivalent” config for Row Filter and Row Filter (Labs) to retrieve rows between row_start and row_end variables:

ROW FILTER

ROW FILTER (LABS)

I’ve uploaded a new copy of the workflow as an example for both.

Row Filter numbering - Equivalent with Labs node.knwf (21.7 KB)

[ipazin]

Hello @takbb,

@takbb

I did a search but only found the article listed, so apologies for not having discovered that already before posting.

No issues. If you hit the wrong path with keyword(s) you can lose hours with search engines. Think we’ve all been there

@takbb

Incidentally is there any publicly available list of known issues that is searchable on a node-by-node basis?

Nop. But this would be nice to have. And not just for bugs but also possible enhancements.

Br,
Ivan