Read multiple files from filesystem in parallel and execute a workflow


I have a directory basically the knime anamoly detection example hosting 16 files.

I want KNIME to read all the 8 files in parallel and then call a workflow to execute certain tasks on it.

After the 8 files are fnished the next 8 should be picked up.The Server is having 8 CPU's and thats the reason i want to pickup 8 files at a time can KNIME do this ?






Hi Hrishy,

in this case you need the paralel chunk loops. They are in the Virtual Loop Extension on Labs.

You would actually need to loops. The outer one is the parallel chunks and the inner one a basic table row to variable loop. The outer one will generate the processes and the inner one takes its chunk of data and read (or process its one by one)

Best, Iris