Trying to setup recursive loop with List Files/Folders Node - possible?

[AustinScott]

So I’m using SMB Connector to the List Files/Folders node to scan for all the files in a subfolder. However, if I list everything, I run into some really random permission errors. I’d love to just ignore those and keep going, but doesn’t look possible by default. Hence, I’m looking at a recursive loop to help.

However, I can’t really start a recursive loop before a List Files and Folders node, since there isn’t a connector on the left side of the list files/folders node. So I tried to use it after the list files/folders node and then before a files/folders meta info node. So it woudl in theory loop through each folder and pull the metadata.

So how would you put the List Files/Folders into a recursive loop?

I thought I was getting somewhere here, but I can’t seem to figure out how to iteratively pass the path to the list folder metadata node.

[mlauber71]

@AustinScott welcome to come to the KNIME forum. These things: you can always use flow variables to force a connection or an iteration and transport information.

On a general note if you read about loops and catching errors you should be able to combine the necessary information.

KNIME, Paths and Loops — Automate Everything

KNIME — Cases, Switches and Catching Errors

More specifically you can for example run a job until it is finished if you do not want to catch and handle errors.

Error Handling within Table Row to Variable Loop Start

@JGebhard I wanted to build a workflow that would do a try catch anyway. It does follow the model and would try to append sheets to an existing Excel file. In the first instance of the outer loop a _temp.table will be written that would contain the procedures and one could follow if the loop was successfil. [image] Within the component an inner loop would try 10 times (you could adapt that) to finish the job. A Try Catch block would check if the operation was successful and fill a Flow Varia…

[takbb]

Hi @AustinScott , I have a component that I modified to uses a recursive loop with a List Files/Folders node as detailed in this post…

@takbb

So I had a rethink, and decided there were some optimisations to be made. With the introduction of a recursive loop and judicious use of List Files/Folders within the loop (which now doesn’t search any subfolder of a folder than contains a workflow.knime file), rather than have List Files/Folders blindly search every single subfolder in the workspace, design/dev effort increases a little but run-time effort is reduced and the performance skyrockets!

Feel free to download and disconnect the component so you can edit it to see how it works. It might give you some pointers too.

[AustinScott]

For some reason Knime wont let me drag this component into the editor. I tried dragging to the main editor and also over into nodes. Nothing happens on my Mac OS (tried chrome and safari). I’ve added some other components before, so I know its possible. Not sure what’s going on.

the Administrator of the folder supposedly gave my account more access, so I’m trying to do this without the loop and error handling mentioned above. If it doesn’t work, i’ll be back to try some fo the methods above again. Thank you both for your time!

[takbb]

Hi @AustinScott , here’s a workflow containing just the component so hopefully you can load that into your KNIME AP. Hope that helps

Recent Workflows component workflow.knwf (49.1 KB)