RowIndex calculated wrongly?

[TotalDataLoss]

I just stumbled across something what i think is a bug. In the preview the RowIndex in a column Expression node for the first row is 0:

Later in the output table it is 1:

Has something changed/gotten ouf order with an update? I looked up an old documentation (of mine) where the screenshot looks like:

I can not say from when exactly the screenshot is nor what thenode was - it’s cut of on top and i don’t remember.

i realized this today because 1 table was 1 row shorter tahn i expected.

[elsamuel]

Different nodes handle row indices differently. This has been noted, and there are open tickets to address the inconsistencies.

See this thread, for example:

Zero-based vs. one-based row indexing

Dear Knimers, The “Table creator”, “Empty Table Creator” RowIDs start with “Row0”, “Row1”, “Row2” To take the first three rows with a Row Filter one has to start with “1” and end with “3”. When taking the first three rows with a Row Filter by using flow variables to set the range, one has to use “0” and “2”! When pointing at the third row in a node output window, the tooltip says “3” The ROWINDEX in the Math, String Manipulation, Java Snippet nodes start with 0, 1, 2 The ROWINDEX in the Rule-…

[TotalDataLoss]

Hi Elsamuel,
Good to hear from you again. That is good to know but in the case i posted is the same node: preview and actual result differ…

Greetings!

[Aswin]

Dear @TotalDataLoss

Yes I see the same thing. Obviously “rowIndex()” is using zero-based indexing when hitting the “Evaluate” button as compared to executing the node; then it changes to one-based indexing.

Resulting table:

Looks like a bug to me. It looks as if at some point during the development of the Column Expressions node rowIndex() was changed from zero-based to one-based, but somehow this correction did not happen for the code that is behind the “Evaluate” button.

Best
Aswin

[TotalDataLoss]

Hi @Aswin, good to know that it is not just on my machine, so i don’t have to get paranoid about error messages i got the last days

In my opinion it would be more reliable if the preview calculation would use the exact same calculation as later on.

Right now i am using a rule based rowfilter which doesn’t have a preview but is there a way to look up what values functions calculate, in Excel you can watch formulas getting calculated step by step or a debugging modus?

[Aswin]

Hi @TotalDataLoss

…is there a way to look up what values functions calculate, in Excel you can watch formulas getting calculated step by step or a debugging modus?

Are you asking this for the Rule-based Row Filter? I am afraid this is not possible without evaluating the whole node and inspecting the output. I think the Column Expressions node is a bit of an exception to the rule in having an “Evaluate on first row” feature. Other nodes that do this in some form are the Python and R nodes, but in general these nodes should only be used if there is no Knime-native way of solving the problem.

Best
Aswin

[takbb]

Hi @elsamuel, do you remember this post re working out a Goal Seek, and initially your workflow had a small issue that you resolved?

Excel Goal Seek function in KNIME

Thanks for pointing that out. At the last minute, I’d modified the expression I used to identify the first row (using the Column Expressions node), and that broke the entire thing. It’s been corrected. I’ve also decreased the step size for the parameter variation (to get closer to the “correct value”), and switched to a Bayseian search strategy to speed things up a bit. The output is now 5.578.

Well on reading this new post, your post came back to me. This now explains it.

The issue was that you initially posted the Column Expression with the following code:

if (rowIndex()== 0) 
    { variable("Volume")
    }
else
{column("Volume")}

Whilst I couldn’t make your workflow produce the correct result, when I looked in the Column Expression, it evaluated correctly on first row, so I couldn’t work out why your code wasn’t working, as it all appeared good!

But the subsequent fix that you then posted was this:

if (rowIndex()== 1) 
    { variable("Volume")
    }
else
{column("Volume")}

That had me quite puzzled at the time because your code appeared to have evaluated ok, and yet here you were now saying it needed to say “== 1” But as it was now working, I figured it was all good and moved on. I just parked it away as “one of those things” for possible future reference.

It turns out … we had stumbled on this bug in Knime, but at that point, we didn’t realise it was Knime itself that had caused this trip up.

Now it all starts to make sense!

[TotalDataLoss]

Hi @takbb : I feel like a therapist, i help solving problems from the past
Glad that now my testing on this was of help for you…

[takbb]

@TotalDataLoss … Yep. I’ll finally be able to sleep easy again. You wouldn’t believe the trauma and anguish this has caused me these past couple of weeks!

But… in all seriousness, you probably have saved me time in some future problem that now won’t happen because I’ll be aware of the issue. So thank you from my future self!

[TotalDataLoss]

I can imagine
Yeah thank you yesterday you helped me out that’s how it goes. And its not solved it was more a hint…