I am trying to iteratively apply the output table of a group of nodes back into them as a new input table for the next iteration, but as far as I can tell, this is not possible (if it is possible, then I'd love to know about it!).
What would be nice is a set of loop nodes which will iterate over a loop, using the output from iteration n-1 as the input for iteration n, until a condition is met (e.g., an iteration produces an empty table). The loop end node could allow output to be concatenated (row-wise or column-wise, depending on what your loop is doing to the data), or (assuming that your completion condition was not an empty table!) just take the last iteration table as the output.
I agree that such a loop would be handy, but as long it is not available there is a workaround to get a similar behaviour by combining varibale connection and external files:
1. create a small workflow that performs a single iteration starting with a csv reader for the input table and ending with a csv writer for the update/processed table at. The writer and reader point to the same file, this is how the "feedback" is realized.
2. create a simple empty loop with just a counting loop start directly connected to a loop end
3. connect the loop start to the csv reader with a dummy variable (red) connection and the csv writer to the loop end respectively. The variable connection create a precedence with the effect that after the loop head, the csv file is read first, modified and overwritten before the loop end node is executed the next time - which basically leads to the desired feedback iteration behaviour
This is just a general concept and would have to be modified to be usable with not trivial string serializable data. I successfully use it with a database backend instead of csv files and a more sophisticated feedback, but this is the basic idea.
I am not sure about the output concatenation you mentioned, but I guess it should be possible to integrate it in such a framework.
luckily exactly what you are describing is already available in the KNIME Ensemble Learning nodes.
The loop set delegating loop start and delegating loop end outputs the input of the loop end at the loop start as long as an nr of iterations is achieved or the loop end gets a datatable having less than a predefined nr of rows.
However the data passed to the first port of the loop end will be collected for the output of the loop end. To get only some of these you could use a row filter.
Anyway, we will think about a renaming of the nodes. If you have a selfexplaining idea for the name, let me know :-)
@nicolas: Actually I did exactly the same as you suggested before I took some time to make the two nodes.
As a follow on question, I have a similar situation now, where within the delegating loop, for each delegation loop iteration I would like to loop through a tablerow to variable loop to set the properties of one of the nodes using a flow variable. Unfortunately, doing this fails with the console error:
WARN WorkflowManager Unable to merge flow object stacks: Stack can't be merged: Conflicting FlowObjects:org.knime.core.node.workflow.FlowLoopContext@d5dae1 vs. org.knime.core.node.workflow.FlowLoopContext@14f684
I tried nesting the loops the other way round to avoid the problem, but, as suspected, this didn't do what I wanted it to.
Thanks for reporting this issue. Can you please shortly sketch how your workflow looks like so that we can reproduce the problem, or can you even attach this flow to your post? Thanks, Thomas
I have attached a fairly contrived version, as I cannot really show the original, but it demonstrates the sort of behaviour.
nested_loops.zip shows what I want to achieve (i.e., for each iteration of the delegating loop, each value in the 'to delete' table is applied succesively, and if this results in an empty row it is deleted)
nested_loops_-_incorrect.zip shows the alternative nesting, which runs, but does not give the desired effect.
Thomas and me are currently trying to solve this problem.
Just one word about the workflow goal. You want in the string column all (sub) parts of your string be deleted which are part of the other string column. And cell which are than empty should be deleted?
the problem, what I currently see, is that the output of the delegating loop end are all iterations of the loop. And you would only need the last output?
