Problem: Invoking a workflow containing a Call Workflow node from command line

[jan_lender]

I introduced nesting workflows to my KNIME project recently. It works fine as long as I run my workflows in a graphical mode of my KNIME Analytics Platform IDE (I use the version 3.7.1, my platform is MacOS Mojave). Yes, I’m still unsure about some details about nesting workflows this way but I’m getting more and more familiar.

Since I started invocating the workflow in from command line in batch mode I’m facing a problem I don’t think I can solve on my own. Well, I have no idea how a workflow knows where to look for the workflow it is supposed to call. There is a path to the workflow in Call Workflow node’s settings.xml file but the path is relative to the root of the current workspace. And I don’t think there is such thing as a workspace when a workflow is invoked from command line. I might be wrong about this but it is the only explanation of the fact all command line invocations of the workflow which calls another one ends with a NullPointerException, I can make out.

These are my workflows I simulated the problem with.

Executing Workflow

Quite simple workflow that just writes Flow Variables to log using logInfo method.

Workflow Invoker

A workflow that calls an inner workflow and performs logging at its begin and its end.

Let’s have look at Call Workflow (Table Based) node configuration.

A successful invocation from the KNIME Analytics Platform IDE

An attempt to invoke the workflow from command line

This is a shell script supposed to invoke the workflow

#!/bin/bash

workflow_name=workflow_invoker

run_identifier=knime_wf_${workflow_mame}_run_$(date +%Y-%m-%d-%H-%M-%S)

current_dir="$PWD"

knime  -nosplash \

-application org.knime.product.KNIME_BATCH_APPLICATION \

-workflowDir="/Users/jan/workspace/ders/ads_poc/knime/elaboration/Nesting/Workflow Invoker" \

-reset \

-consoleLog \

-destDir="$current_dir/out/${run_identifier}-out/" \

-vmargs -Dorg.knime.core.maxThreads=2 -Xmx4g

The execution has failed. There are all log entries the Workflow Invoker workflow produced between its two Java Snippet nodes:

2019-05-02 12:57:17,580 : DEBUG : KNIME-Worker-1 : Node : Java Snippet : 0:314 : Configure succeeded. (Java Snippet)

2019-05-02 12:57:17,581 : DEBUG : KNIME-Worker-0 : WorkflowManager : Call Workflow (Table Based) : 0:311 : Call Workflow (Table Based) 0:311 doBeforePreExecution

2019-05-02 12:57:17,581 : DEBUG : KNIME-Worker-0 : NodeContainer : Call Workflow (Table Based) : 0:311 : Call Workflow (Table Based) 0:311 has new state: PREEXECUTE

2019-05-02 12:57:17,581 : DEBUG : KNIME-Worker-0 : WorkflowManager : Call Workflow (Table Based) : 0:311 : Call Workflow (Table Based) 0:311 doBeforeExecution

2019-05-02 12:57:17,581 : DEBUG : KNIME-Worker-0 : NodeContainer : Call Workflow (Table Based) : 0:311 : Call Workflow (Table Based) 0:311 has new state: EXECUTING

2019-05-02 12:57:17,581 : DEBUG : KNIME-Worker-0 : WorkflowDataRepository : Call Workflow (Table Based) : 0:311 : Adding handler ca59ec15-a946-4e1b-9a6a-18a5def98b24 (Call Workflow (Table Based) 0:311: <no directory>) - 3 in total

2019-05-02 12:57:17,581 : DEBUG : KNIME-Worker-0 : LocalNodeExecutionJob : Call Workflow (Table Based) : 0:311 : Call Workflow (Table Based) 0:311 Start execute

2019-05-02 12:57:17,643 : DEBUG : KNIME-Worker-0 : Node : Call Workflow (Table Based) : 0:311 : reset

2019-05-02 12:57:17,643 : ERROR : KNIME-Worker-0 : Node : Call Workflow (Table Based) : 0:311 : Execute failed: ("NullPointerException"): null

2019-05-02 12:57:17,644 : DEBUG : KNIME-Worker-0 : Node : Call Workflow (Table Based) : 0:311 : Execute failed: ("NullPointerException"): null

java.lang.NullPointerException

at org.knime.workbench.explorer.ExplorerURLStreamHandler.resolveMountpointRelativeUrl(ExplorerURLStreamHandler.java:272)

at org.knime.workbench.explorer.ExplorerURLStreamHandler.resolveKNIMEURL(ExplorerURLStreamHandler.java:200)

at org.knime.explorer.nodes.callworkflow.local.LocalWorkflowBackend.newInstance(LocalWorkflowBackend.java:159)

at org.knime.productivity.callworkflow.table.WorkflowBackendFactory.create(WorkflowBackendFactory.java:104)

at org.knime.productivity.callworkflow.table.CallWorkflowTableNodeModel.executeInternal(CallWorkflowTableNodeModel.java:117)

at org.knime.productivity.callworkflow.table.CallWorkflowTableNodeModel.access$0(CallWorkflowTableNodeModel.java:112)

at org.knime.productivity.callworkflow.table.CallWorkflowTableNodeModel$1.call(CallWorkflowTableNodeModel.java:99)

at org.knime.productivity.callworkflow.table.CallWorkflowTableNodeModel$1.call(CallWorkflowTableNodeModel.java:1)

at org.knime.core.util.ThreadPool.runInvisible(ThreadPool.java:615)

at org.knime.productivity.callworkflow.table.CallWorkflowTableNodeModel.execute(CallWorkflowTableNodeModel.java:96)

at org.knime.core.node.NodeModel.executeModel(NodeModel.java:567)

at org.knime.core.node.Node.invokeFullyNodeModelExecute(Node.java:1186)

at org.knime.core.node.Node.execute(Node.java:973)

at org.knime.core.node.workflow.NativeNodeContainer.performExecuteNode(NativeNodeContainer.java:559)

at org.knime.core.node.exec.LocalNodeExecutionJob.mainExecute(LocalNodeExecutionJob.java:95)

at org.knime.core.node.workflow.NodeExecutionJob.internalRun(NodeExecutionJob.java:179)

at org.knime.core.node.workflow.NodeExecutionJob.run(NodeExecutionJob.java:110)

at org.knime.core.util.ThreadUtils$RunnableWithContextImpl.runWithContext(ThreadUtils.java:328)

at org.knime.core.util.ThreadUtils$RunnableWithContext.run(ThreadUtils.java:204)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at org.knime.core.util.ThreadPool$MyFuture.run(ThreadPool.java:123)

at org.knime.core.util.ThreadPool$Worker.run(ThreadPool.java:246)

2019-05-02 12:57:17,646 : DEBUG : KNIME-Worker-0 : WorkflowManager : Call Workflow (Table Based) : 0:311 : Call Workflow (Table Based) 0:311 doBeforePostExecution

2019-05-02 12:57:17,646 : DEBUG : KNIME-Worker-0 : NodeContainer : Call Workflow (Table Based) : 0:311 : Call Workflow (Table Based) 0:311 has new state: POSTEXECUTE

2019-05-02 12:57:17,646 : DEBUG : KNIME-Worker-0 : WorkflowManager : Call Workflow (Table Based) : 0:311 : Call Workflow (Table Based) 0:311 doAfterExecute - failure

2019-05-02 12:57:17,647 : DEBUG : KNIME-Worker-0 : NodeContainer : Call Workflow (Table Based) : 0:311 : Java Snippet 0:314 has new state: CONFIGURED

2019-05-02 12:57:17,647 : DEBUG : KNIME-Worker-0 : Node : Call Workflow (Table Based) : 0:311 : reset

2019-05-02 12:57:17,647 : DEBUG : KNIME-Worker-0 : Node : Call Workflow (Table Based) : 0:311 : clean output ports.

2019-05-02 12:57:17,647 : DEBUG : KNIME-Worker-0 : WorkflowDataRepository : Call Workflow (Table Based) : 0:311 : Removing handler ca59ec15-a946-4e1b-9a6a-18a5def98b24 (Call Workflow (Table Based) 0:311: <no directory>) - 2 remaining

2019-05-02 12:57:17,647 : DEBUG : KNIME-Worker-0 : NodeContainer : Call Workflow (Table Based) : 0:311 : Call Workflow (Table Based) 0:311 has new state: IDLE

2019-05-02 12:57:17,648 : DEBUG : KNIME-Worker-0 : Node : Call Workflow (Table Based) : 0:311 : Configure succeeded. (Call Workflow (Table Based))

2019-05-02 12:57:17,648 : DEBUG : KNIME-Worker-0 : NodeContainer : Call Workflow (Table Based) : 0:311 : Call Workflow (Table Based) 0:311 has new state: CONFIGURED

2019-05-02 12:57:17,716 : DEBUG : KNIME-Worker-0 : Node : Java Snippet : 0:314 : Configure succeeded. (Java Snippet)

And here is an error pop-up

I’ve found some related topic here.

Call Local Workflow consistently fails from command line, when producing report output

I have a workflow (minireport_test) that runs fine at the desktop. It calls a local workflow, which produces a report (_generate_aMiniReport, taken from Examples site). It uses Call Local Workflow and Binary Object to File nodes. When ran from Windows command line… "C:\Program Files\KNIME\knime.exe" -reset -consoleLog -nosplash -noexit -application org.knime.product.KNIME_BATCH_APPLICATION -workflowDir="d:/Users/myuser/knime-workspace/minireport_test" it fails and produces this output… -----…

and here

Call Local Workflow Node from "main" workflow

Hi all! What I’m trying to do is to build “main” workflow in which I will call other workflows using Call Local Workflow Node so I have couple of questions and your ideas are welcomed Is this ok approach or the is a better way to do it? Maybe using Metanodes? As Call Local Workflow Node needs input (and I actually don’t need it for now) what node is best to use to start a workflow? I used Timer Info. The same node when started shows progress but doesn’t show any other percentag…

The second one of the topics above suggests it is not a good idea to use the call-workflow approach if the intention is to run workflows from command line. Is that true?

[jan_lender]

Well, I’ve just solved it obviously trying using a relative path to the nested workflow in a Call Workflow node.

It was quite clear after some attempts when an error message had informed me that Workflow Path must begin with “/” or “…”. Those two dots were easy to understand.

[ipazin]

Hi Jan!

@jan_lender

And I don’t think there is such thing as a workspace when a workflow is invoked from command line

Workspace is started when workflow is called in batch mode. This can be seen if you delete your workspace (Do not try this at home if you do not have a backup!) and then call a workflow from command line as it will create a workspace

@jan_lender

The second one of the topics above suggests it is not a good idea to use the call-workflow approach if the intention is to run workflows from command line. Is that true?

Not sure. I carried out my project using call-workflow approach with batch mode. Development is harder and troubleshooting is a challenge but workflow are still running every day. At least I do hope so

Additionally take a look at this topic:

Nullpointer Error When Executing "Call Local Workflow" in Batch Mode

Hi Folks, I am trying to execute a simple workflow that only has 2 nodes: Json Input node and Call Local Workflow (Row Based) I got this error in batch mode: [image] Here is the command I executed: ./knime -nosplash -application org.knime.product.KNIME_BATCH_APPLICATION -workflowDir="/Users/username/Desktop/knime-workspace/CallWorkflow" -reset Knime version: 3.6.2 OS: MacOS High Sierra Thanks in advance! -H

Br,
Ivan

[peleitor]

I’ve had some issues running KNIME 3.6 in linux from command line and call workflow. I downgraded to 3.5.3 and it worked fine. Not sure if this helps your case, but maybe you can give it a try.
Cheers