I have multiple workflows that needs to be completed first before going to the next node or task (actually i separated it but design wise, it should be joined/integrated). The ff workflows are data ingestion that needs to be done first before running/calling a stored procedure to process them.
What node to be used in order to identify a completed task while waiting for the other wf? Hope you have some example similar to my question.
You could wait for a file to have changed on the disk
The easiest way might be to just call all the workflows you want to finish before you continue.
An approach I employ on a regular basis on a big data system where there might be several dependencies is to execute a workflow via the knime server every hour (or so) and check which tasks have been finished and which databases are ready (wi
This new data) and then execute the following tasks. If nothing is there to do the job will rest after a few seconds. This makes for a quite robust setting if you continue to add tasks in a fast developing environment (dependencies over the whole system are hard to track so each sub task has to make sure on its own that it is fit to run).