Moving downloaded file to folder using Transfer files

Hi team,

I have made a simple workflow which goes on a website and download reports.

For eg I have 20 Entity. For 20 Entity I need to download 14 reports.
My plan is to

  • Use chunk loop to download 14 reports of 1 entity

  • Using Table row to variable loop twice , I am able to split the 14 reports into two parts to speed up the downloading of the reports from the browser

  • Once 1 Table row loop finish move all the downloaded report to the entity folder

  • Once 2 Table row loop finish move all the downloaded report to the entity folder

  • The Entity folder is created by Transfer files Node

The change which I am facing I am not sure how to fit the transfer files node after the first Loop end and 2 Loop end. When I execute the last loop end(For Chunk Loop), then Transfer files is executed instantly but as the files are not downloaded yet so nothing moves to the correct directory.

The time taken to download the reports in variable and depends on the data in report which is unknown.

@Ankit_smart I would try and split the task into several consecutive loops. If you have to transfer information between them maybe store it a a table and reuse it later.

If you want to do the downloads in parallel to speed them up you could use a parallel execution - that should work if the server in question would allow that.

Could you merge the variables of the two loop end nodes and then connect those variables to the tranfer files node?
br

Hi Mlauber,

Yes, putting several consecutive loop is an option, I was not doing it due to the parallel execution. Can you please guide how I can do a parallel execution in this case. The server does allows parallel execution.

Yes, I would very much like to do that but can you please guide me how I can do this.

i.e. How can I connect to the First loop end to file transfer files and then finish the loop.

There is a merge variables node so you could connect the loop end(s) with their variable ports to the merge variable and then that merge variable node to the transfer files
br