How to close two loops at once

[Thoralf]

I want to append a column to a table row by row, because the pivot column will look different in each row. I have therefore started two Chunk Loops. But after the Column Appender I can only close one loop. I thought this is exactly what the node Loop End (Column Appender) is for.

But I get the error message: Can’t merge FlowVariable stacks! (likly a loop problem). What can I do? How do I close two loops at once?

[takbb]

Hi @Thoralf , without actually seeing how your workflow is configured, its difficult to determine if your workflow would do as you intend.

Are both those chunk loops set to 1 row chunks? Currently even if the workflow loops could be terminated they way you appear to want, they’d be attempting to perform a cross join, so every row from XPath would be matched to every other row. i.e. If the output from our XPath is 10 rows then the output at the end of the loop would be 100 rows. Is that what you are expecting?

As with regular programming languages, loops in KNIME cannot join together part way through. With two loops working together one loop must be fully contained inside the other.

So, when using two loops you need to decide which is the “inner” loop and which is the “outer” loop. The inner loop will be iterated entirely for each execution of the outer loop. You then need to also include a loop-end for both loop-starts.

Re the error that you see: the specific problem you face is that the two CHUNK LOOP START nodes are on separate branches. When you split branches you also split the Flow Variable stack. Trying to bring these together again later causes the error “Can’t merge FlowVariable stacks!” because you now have two totally separate loops that you are now trying to “interleave” which is “meaningless” from a programming perspective.

(Imagine I’m working through two packs of playing cards simultaneously, and have to read out the cards, in what sequence should I read the cards from each pack? Should it be 1 from each sequentially? Two from one and then five from the other, then ten from the first and one from the second? All from one and then all from the other? It is undefined.)

I have mocked up a similar loop construct in this screenshot:

The immediate fix to the error is to use a red flow variable connection tells KNIME to treat one branch as the “inner loop” and you would then need to terminate both loops.
e.g. one of the following:

But I have a feeling that twin loops is not actually what you really want to use for this problem, and without actually seeing some data and configuration, I cannot say for sure. In the above loops, for each iteration of the “outer loop”, the inner loop will be executed in its entirety. So you effectively cross-join the result.

If that isn’t what you want, it is possible for example that one of the following “patterns” might be better, but again whether they would do what you are aiming for, I cannot tell.

If you can upload a simplified workflow with demo data to demonstrate what you have as inputs and what your expected outputs are, somebody would be able to give more specific help.

[Thoralf]

Hey @takbb,
Thank you very much for your detailed description of the problem. Somehow I had hoped that KNIME would offer a way to avoid nesting loops. But your explanations make perfect sense.

I have tried your examples and either run into errors or in a never-ending execution process. But this is probably more a problem with my data structure.

The best thing is that I found a solution without loops by trying it out. So you have indirectly helped me a lot Thanks again

[takbb]

Hi @thoralf, you’re welcome. Yeah I wasn’t necessarily expecting any of the loop solutions to be exactly right for you but hopefully can give a guide to possible constructs to consider.

If you managed to achieve what you needed without using loops at all, then that is so much the better as loops will (nearly?) always be slower than non-loop solutions although they’re “sometimes” easier to understand