After a lot of trial and error I got a new workflow to at least start in batch mode. Note that the workflow works 100% fine when run inside KNIME AP GUI.
However at some point the workflow fails. When I open the failed workflow status in KNIME GUI after a List Files node:
I see this as output after Path to String:
The Path column is missing but yet somehow Path to String worked correctly? This was very confusing and I thought it is the root cause of the error but I suspect it’s just a weird artifact of batch mode.
With the String in Location column I generate a Destination folder (to use in Transfer files later):
Here very similar behavior. After String to Path the new Path column is all missing values even though the source string column is correct:
The issue then happens here:
If in Batch Mode the Path and Destination columns and hence flow variables actually were missing the Loop Start node should instantly fail but it doesn’t. I can see that the variables are actual set so the missing values in the Table are indeed some artifact.
At the transfer Files node I get a an java.nio.file.NoSuchFileException.
This is for a file that was just before listed with the List Files node. When I try to find the file it indeed does not exist anymore!!! I do not want to move the files, I want to copy them:
And this works 100% fine in GUI mode.
The workflow (Transfer File node?) in Batch mode seems to somehow silently delete files!!!
Now comes the extreme irony of this issue. I wanted to create a workflow that automatically backs-up all my workflows into a .knar file. Again this works perfectly in the GUI! But in batch it actually deleted a file of a workflow. Here the full error log:
WARN KNIME-Worker-3-Transfer Files 0:106:87 PathCopier Something went wrong during the copying / moving process. See log for further details.
java.nio.file.NoSuchFileException: C:\Users\kienerj\knime-4-workspace\ATest Log.artifacts\openapi-input-parameters.json
That file is now gone:
It’s not an important one and gets restored automatically. But since I ran the workflow in batch mode a few times it kept deleting the first file encountered leading to a node being removed from the workflow due to removing settings.xml. Luckily it is an unimportant test workflow.
I have attached the full workflow. To run it in the GUI, one need to change the Configuration of the “Input and Prepare component”, to run it in Batch Mode I used this command (Note: Windows 10):
“C:\Program Files\knime4\knime.exe” -workflow.variable=destination_folder,“G:\My Drive\KNIME”,String -workflow.variable=workspace,“C:\Users\kienerj\knime-4-workspace”,String -workflow.variable=backup_name,knime_newest_backup,String -workflow.variable=date_format,yyyy-MM-dd,String -reset -consoleLog -noexit -nosplash -application org.knime.product.KNIME_BATCH_APPLICATION -workflowDir=“C:\Users\kienerj\knime-4-workspace\Backup Workflows (Batch Mode)”
Again the workflow variables need to be changed to make it work and obviously please set workspace variable to some test workspace!!!
Again it seems the Transfer Files Node is deleting files in Batch mode on Windows 10 even if “Delete Source” is unchecked (the default).
Backup Workflows (Batch Mode).knwf (99.5 KB)
Can you please investigate this severe issue? Thanks you!