Execute and export multiple workflows from a wrapper workflow

Hi folks - good discussion. I wanted to jump in with my own “Call Workflow” use case.

I develop example workflows for my users (many of whom are beginners). They appreciate opening these example workflows in a finished state so they can immediately see the results from each node as they explore and learn. Forcing users to re-run example workflows that start in a reset state is not always feasible as connections to external resources are not always available or are complicated to setup.

Unfortunately I sometimes need to re-run these example workflows myself because some of the underlying nodes have been updated. Until now, that has been a manual process. But as I’ve now got about 70 example workflows this is becoming a burden.

I’d really love there to be a way for a KNIME “main” workflow to:

  1. Reset a named workflow
  2. Run it
  3. Log the results (pass or fail)
  4. Save it in the completed state
  5. Export it as a zip file
  6. Move on to the next workflow

If the “Call Local Workflow” could reset and save then I’d be really close. I think “exporting” is really just “save as zip file and rename extension”. I’m not sure about logging pass/fail results but this might be achieved in other ways.

The main advantage of the Call Local/Remote nodes is, that if each row needs to generate a report, this will be possible. You can’t however generate multiple reports from a single workflow. That is how I use these nodes. This helps greatly for automation. Users can drop files into a defined shared folder and the workflow will pick them all up (List files), process them and generate a report (sent by email).

If you don’t need reporting the metanode way is probably easier and more efficient.

1 Like

Hi there!

interesting case. I don’t see Call Local Workflow have these abilities, but to reset, run and save workflow after execution I recommend to run workflow in batch mode (https://www.knime.com/faq#q12). To run all 70 workflows one by one in batch mode is something you need to see how to accomplish. Maybe use KNIME! KNIME calls KNIME in batch mode :slight_smile:

Br,
Ivan

Thanks ipazin:

I knew about the command line batch mode and, though I was skeptical, used it as the basis of the solution I put together yesterday. But I was very pleased with the results!

I build a simple batch file using the list of workflow directories and the help of Excel. The batch moved nicely from workflow to workflow, running and saving each. If the workflow didn’t run to completion then it would simply pause and pop-up a GUI window with more details of the problem. I think there is a way to prevent the pause, but pausing was exactly what I wanted. I could acknowledge the problem and then allow the batch to move on.

The command I used for each workflow (running on Windows) was:

"C:\Program Files\KNIME\knime.exe" -consoleLog -nosplash -reset -application org.knime.product.KNIME_BATCH_APPLICATION -workflowDir="D:\knime\workspace\{workflow_name}"

But watch out for a small gotcha. After running, there is a long pause while the workflow is being saved, but it is not obvious that anything is happening. After killing the paused terminal I ruefully discovered that the half of my workflow that had not been saved had disappeared and I needed to restore it from a backup.

2 Likes

Hi there!

Glad it worked. To some point at least. And again interesting :slight_smile:

This long pause is at the end? Because I guess the saving part goes after every workflow so why do you think it is connected to saving?

Br,
Ivan

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.