Python Source node works in GUI, fails in batch mode

KNIME Extensions
,
1

Hi everyone, looking for some insight. I’m running Knime 4.0.2 on Windows 10, using the Python Source node. I have Anaconda installed (python 3.6.9, no python 2.x), set up a custom environment using the Knime Python configuration dialog, and have the custom environment selected as the default environment for Python 3. In addition, I have Python 3 configured as the default version of python for Knime to use.

Everything works as expected inside of the GUI environment - the python source node executes perfectly. However, when I run the same workflow in batch mode using this command:

& “C:\Program Files\KNIME\knime.exe” -noexit -consolelog -nosplash -reset -nosave -application org.knime.product.KNIME_BATCH_APPLICATION -workflowDir=“/path/to/workflow”

I receive an exit code of 4, along with the error below, and the workflow outputs fail to materialize.

org.knime.python2.kernel.PythonIOException: Could not start Python kernel. Error during Python installation test: Python installation could not be determined.Could not find conda environment:
You can list all discoverable environments with conda info --envs.

    at org.knime.python2.kernel.PythonKernel.testInstallation(PythonKernel.java:378)
    at org.knime.python2.kernel.PythonKernel.<init>(PythonKernel.java:253)
    at org.knime.python2.nodes.source.PythonSourceNodeModel.execute(PythonSourceNodeModel.java:86)
    at org.knime.core.node.NodeModel.execute(NodeModel.java:733)
    at org.knime.core.node.NodeModel.executeModel(NodeModel.java:567)
    at org.knime.core.node.Node.invokeFullyNodeModelExecute(Node.java:1192)
    at org.knime.core.node.Node.execute(Node.java:979)
    at org.knime.core.node.workflow.NativeNodeContainer.performExecuteNode(NativeNodeContainer.java:559)
    at org.knime.core.node.exec.LocalNodeExecutionJob.mainExecute(LocalNodeExecutionJob.java:95)
    at org.knime.core.node.workflow.NodeExecutionJob.internalRun(NodeExecutionJob.java:179)
    at org.knime.core.node.workflow.NodeExecutionJob.run(NodeExecutionJob.java:110)
    at org.knime.core.util.ThreadUtils$RunnableWithContextImpl.runWithContext(ThreadUtils.java:328)
    at org.knime.core.util.ThreadUtils$RunnableWithContext.run(ThreadUtils.java:204)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at org.knime.core.util.ThreadPool$MyFuture.run(ThreadPool.java:123)
    at org.knime.core.util.ThreadPool$Worker.run(ThreadPool.java:246)

Based on the first line of the error message, my naive assumption is that batch mode is either not respecting the Python3 default setting (and trying to use the unconfigured python 2 environment), or not respecting the custom environment choice for Python 3.

I’ll add that removing the Python Source node from the workflow allows the workflow to run in batch mode successfully.

Any ideas? I’d love to get this workflow running unattended.

2

Hi jim-christopher,

Welcome to the forum!

I believe you need to point the batch executor to your KNIME preferences. The reason is, that in batch mode, there is no notion of “workspace”, which is where the preferences are usually stored. It can be done like so:

  1. Export your preferences via File > Export Preferences…
  2. Add the option -preferences=“<path-to-exported-preferences-file>” to your batch mode command

Hope that works!

Marcel

5 Likes
3

Hello MarcelW!

Appreciate the warm welcome. You definitely solved my problem too - exporting the preferences and adding the command-line option got everything working smoothly! Thank you so much - I feel like I missed this in a FAQ someplace, I appreciate your patience :slight_smile:

4 Likes
4

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.