Looping without concatenating results?

Is there a solution for looping through a workflow but not concatenating results like the normal Loop End does? I’m looping through 50+ very large files and while my computer can handle each one individually, when you sum them all up, it’s over 70,000,000 rows. I’ve noticed that after a while the workflow becomes very slow, and I suspect it’s because KNIME is holding onto more data than it needs and is running low on memory.
To be clear my process is this:
[loop start] – [input file] – [run analysis] – [save result to .table] – [loop to next file]

Right now I have it built with a standard “Loop End” which of course holds onto all the data. Because I’m saving the results to a .table before looping to the next file, I don’t need the data concatenated in the Loop End node.

Thanks!

Hi Snowy,

just use the Variable Loop End instead:

8 Likes

Thanks! That’s exactly what I was looking for :slight_smile: Just did a test with limiting the rows in my data set and it worked beautifully.

Snowy

3 Likes

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.