In all my workflows, “loop end” node, fills up a huge table with all iterations and slows down the later iterations. Is there any way to empty this table or disregard it?
Hi,
I would suggest using a Variable Loop End instead and write the data to disk using a CSV Writer with the append option enabled.
Kind regards
Alexander
2 Likes
Hi AlexanderFillbrunn,
I may be new to Knime. I thought a Variable Loop End is for looping variables. In my workflow I am looping columns or files.
Also, a CSV Writer with the append option enabled would create a building up CSV file. I am saving to multiple csv files inside the loop, and I would not like a huge CSV file. Perhaps, you are implying something that I do not know. Could you please clarify?
Thanks
Hi,
If your loop end is taking a long time and consuming a lot of memory, the best option is to write the table to disk. You can do that using the CSV Writer. The Variable Loop End is necessary because you need some loop end. When the loop is done you can read in the CSV again.
Kind regards
Alexander
1 Like
Thank you for your reply. I am not sure if I follow you, so I have implemented and attached here. Would you mind if you continue by modifing it, please?
YourSuggestion.knwf (13.5 KB)
Hi,
I am on mobile right now and cannot continue the workflow. The Variable Loop End comes after the CSV Writer. You can connect it by right-clicking the CSV Writer and select “Show flow variable ports”. Then you can use the top right red port to connect to the Variable Loop End. Finally, connect the reader to the writer and select append in the configuration. Make sure not to write the column header, as that would result in multiple headers being written. If necessary, you can write only the header in front of the loop first.
Kind regards
Alexander
1 Like
Thank you, it does work now. However, it is slow. About a file per 10 seconds, and later it takes more time.
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.