I have seen examples of loop workflows for people who want to read several files and combine (concatenate) the data from all of them. In my case, I want to read the rows in each CSV file, add some new columns to row, and then write the file in a subdirectory – one at a time. But instead when looping, KNIME combines all of the rows from all the files into one output file.
What is the setting to prevent concatenation?
@njacknis from what I understand you might have to use a different loop end. Instead of collecting you might want to write your new files within the loop and give them different names.
You can use knime like Lego and build the jobs as you need them but you might have to do some planning
2 Likes
I think you may want to do it that you get the paths for the files and then loop over the list of paths, do your processing, create the new path for saving the result, save the result and then end the loop.
Prototype:
loopthroufiles.knwf (110.6 KB)
Overview
Comments:
- variable expressions is used to create a new path based on the old path
- Excel Reader/Writer use Flow Variables to define where files are read from and written to
2 Likes
Hi @njacknis, as per @mlauber71 's comment, you can use an alternative loop end node according to your needs.
Taking @MartinDDDD 's example, if you really do not wish to retain (concatenated) data beyond the loop, then terminating the loop using Variable Loop End instead of Loop End is one solution. In this case, you would simply run the data flow from the “Processing” metanode (or alternative nodes) into the Excel Writer as shown in the @MartinDDDD 's overview, and then the flow variable from Excel Writer would connect to the “Variable Loop End” node, to terminate the loop.
3 Likes
Thank you very much! This worked perfectly. My mistake was checking Folders, not Files in the List Files/Folderss node.