Build a loop that exits when it no longer changes the length of the table

I have a workflow in KNIME that looks for certain attributes of adjacent rows and removes rows that are too similar to the row before them. Hence going in the table might be 5,800 rows and going out the table might be 5,740 rows. I repeat this process until the count of rows at the start is the same as the count at the end - hence no rows were removed.

Is there a way to set up a loop that will handle this? Right now I keep adding more nodes to my workflow until I reach the condition where rows_in == rows_out, it would be advantageous to have a loop handle this for me. I looked at “Table Row to Variable” but that doesn’t quite do it (unless I do some additional math to add the table length to a given row before I start the loop).

This is of course not the same as doing X number of loops where X is the number of rows; I generally do far fewer than that many loops. While it wouldn’t hurt to keep doing more loops after the condition is met, it would be a waste of time to do 5,800 loops when I only need to do 6.

Any suggestions would be great.

Thank you!

My current workflow - which seems to only remove rows once regardless of how many iterations it goes through:
Math Formula (to count the initial rows)
Generic Loop Start
Several math processes to identify which rows to remove, and remove them
Math Formula (to count the rows after the above processes)
Math Formula (to subtract the new count from the initial count)
Table Row to Variable (takes the row count difference and makes it into a flow variable for the loop close)
Variable Condition Loop End

The last Math Formula node is connected to the input of Variable Condition Loop End; while the output of Table Row to Variable is connected to the same Variable Condition Loop End as a flow variable. I then set the Variable Condition Loop End to watch available flow variable “row_change” until it equals 0 and end the loop when that happens. The result unfortunately is that the table shrinks once and then never again; the difference never changes from the difference between the original table and the result of the first iteration. It seems as if the loop is reloading the original table instead of loading the table that was created by the iteration.

I just added another Math Formula node to the start of the loop that counts the rows in the table; it confirms that the table is being reloaded from its original state instead of receiving the result of the steps in the loop. How do I correct this?

I think you are looking for the recursive loops created by Iris. There were a demo related to that too.

Thank you for the suggestion, but I can’t come up with a way to apply it to what I am trying to do. In my case I don’t know how many times I will need to run the loop, I need it to run until the difference in row count between successive runs is 0. I’ve tried this with a recursive loop but it runs infinitely; I tried using a variable condition loop (where I set a variable to pass “true” if the row count difference described above is met) but that doesn’t work correctly either.

I’m not sure what I’m doing wrong here. I could have written this in Perl in about 10 minutes but I’ve been running in circles on this in KNIME for days now.

With the recursive loop, try using a Reference Row Filter to see how many rows from the start of the loop (i.e. the output table of the Loop Start node) remain immediately before the loop end. Use Extract table dimensions and then Java Edit variable to set s String variable as ‘true’ is the number of rows is 0, and ‘false’ otherwise. then use this flow variable to control the ‘End loop’ setting in the recursive loop end

Steve

I finally got it to work with a chain of “Row Filter” -> “Rule Engine” -> “Table Column to Variable”; then passing that variable on to “Recursive Loop End”. It was not the most intuitive thing to put together but it works well now. I ran in to a large number of ways that did not work before then :slight_smile:

3 Likes

hi, I’m facing the same problem with you. Could you please attach the mentioned workflow?

My situation is do recursive looping until the current iteration’s table length is equal to the table length of the previous loop.

Thanks in advance!

Hello @penghor315,

and welcome to KNIME Community!

There is Extract Table Dimension node which you can use to extract number of rows for two output tables you want to compare. This node outputs table dimensions in both table output and as flow variables. As KNIME can not have two flow variables with same name in one node (and you need to bring those two flow variables together to compare two table lengths) you need to rename at least one of them (you can use Rule Engine Variable node for example) and then follow it with Merge Variables node. After that again use already mentioned Rule Engine Variable node to compare two flow variables and output “true” in new flow variable if values are equal to stop Recursive Loop End.

Give it a try and if it’s a bit too much I can make you workflow example.

Br,
Ivan

3 Likes