Execute failed: Failed to initialize Python gateway.

[falaimo]

Hello everyone, sometimes, when running a workflow that uses a Python script, I encounter the error: Execute failed: Failed to initialize Python gateway. I suppose it might be related to some timer, and that it could be extended. Does anyone have any ideas about this? I couldn’t find anything in the Preferences. Thank you."

[bwilhelm]

Hi @falaimo,
I searched for the error and found it for the code for Python-based node extensions: Code search results · GitHub. There should be more information about the cause of the issue in the stack trace which you can find in the KNIME log. Can you please post them here?

[falaimo]

Hi, Benjamin, sure. There is the error stack trace:

2024-04-04 16:20:46,482 : ERROR : KNIME-Worker-74-SARIMA Learner 3:436:341 : : Node : SARIMA Learner : 3:436:341 : Execute failed: Failed to initialize Python gateway.

java.lang.IllegalStateException: Failed to initialize Python gateway.

at org.knime.python3.nodes.PurePythonExtensionNodeProxyProvider.createPythonNode(PurePythonExtensionNodeProxyProvider.java:157)

at org.knime.python3.nodes.PurePythonExtensionNodeProxyProvider.getExecutionProxy(PurePythonExtensionNodeProxyProvider.java:104)

at org.knime.python3.nodes.DelegatingNodeModel.lambda$3(DelegatingNodeModel.java:149)

at org.knime.python3.nodes.DelegatingNodeModel.runWithProxy(DelegatingNodeModel.java:207)

at org.knime.python3.nodes.DelegatingNodeModel.execute(DelegatingNodeModel.java:149)

at org.knime.core.node.NodeModel.executeModel(NodeModel.java:588)

at org.knime.core.node.Node.invokeFullyNodeModelExecute(Node.java:1297)

at org.knime.core.node.Node.execute(Node.java:1059)

at org.knime.core.node.workflow.NativeNodeContainer.performExecuteNode(NativeNodeContainer.java:595)

at org.knime.core.node.exec.LocalNodeExecutionJob.mainExecute(LocalNodeExecutionJob.java:98)

at org.knime.core.node.workflow.NodeExecutionJob.internalRun(NodeExecutionJob.java:201)

at org.knime.core.node.workflow.NodeExecutionJob.run(NodeExecutionJob.java:117)

at org.knime.core.util.ThreadUtils$RunnableWithContextImpl.runWithContext(ThreadUtils.java:367)

at org.knime.core.util.ThreadUtils$RunnableWithContext.run(ThreadUtils.java:221)

at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)

at java.base/java.util.concurrent.FutureTask.run(Unknown Source)

at org.knime.core.util.ThreadPool$MyFuture.run(ThreadPool.java:123)

at org.knime.core.util.ThreadPool$Worker.run(ThreadPool.java:246)

Caused by: java.net.ConnectException: The connection to the Python process timed out. The current timeout is 30000 milliseconds. You can set a longer timeout via the knime.python.connecttimeout system property.

at org.knime.python3.DefaultPythonGateway.waitForConnection(DefaultPythonGateway.java:238)

at org.knime.python3.DefaultPythonGateway.(DefaultPythonGateway.java:192)

at org.knime.python3.DefaultPythonGateway.create(DefaultPythonGateway.java:144)

at org.knime.python3.FreshPythonGatewayFactory.create(FreshPythonGatewayFactory.java:75)

at org.knime.python3.QueuedPythonGatewayFactory$PythonGatewayQueue.enqueueGateway(QueuedPythonGatewayFactory.java:301)

at org.knime.python3.QueuedPythonGatewayFactory$PythonGatewayQueue.lambda$1(QueuedPythonGatewayFactory.java:267)

at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.base/java.lang.Thread.run(Unknown Source)

I’m using KNIME 5.2.3 and python bundled version.
Thank you in advance.
Francesco

[steffen_KNIME]

Hi Francesco @falaimo,

could you maybe give us a minimal workflow so that we can reproduce the issue?

Thanks
Steffen

[aliasghar_marvi]

Hello @falaimo ,

Looks like the SARIMA Learner node from the Time Series Extension is failing. I re-ran the testflows on KAP 5.2.3 and everything seems to run fine.
Like Steffen pointed out, can you share a sample workflow to look into this? And a screenshot too for it.

The error you see normally arises when the node configurations are breaking somewhere.

Thank you

Best,
Ali

[falaimo]

Hi @steffen_KNIME and @aliasghar_marvi,
I can’t share the workflow because I’m developing it for my company to solve a business problem, but I can send you an image to show you its structure. Essentially, a loop loads six features that evolve over time. With the SARIMA node, I make a prediction for the last 8 hours, referencing an 8-day period and a seasonality that evolves over 24 hours/7 days. The purpose of the workflow is to compare the prediction with the actual data to check for discrepancies. The loop is necessary to repeat this process for each feature and for all the sources that generate the data, which are numerous. The workflow typically gets stuck on the SARIMA node due to the known issue, but I don’t believe it’s due to logical flow problems.

[steffen_KNIME]

Dear Francesco @falaimo,

thanks for the details.

We cannot reproduce that issue unfortunately.

Some question which might bring us closer to a solution

1. Is there maybe something else around that error in the KNIME Log file?
2. If you use that node outside of the loop, does it work then (reliably)?
3. You said it sometimes fails. Could you say how often?
4. maybe @bwilhelm has another suggestion how we might determine the issue

I talked to the maintainer, maybe he has time to implement some more progress details in the node showing at which computation step it gives this error message of yours.

Best regards
Steffen

[falaimo]

Hi @steffen_KNIME and @aliasghar_marvi.
Good news: I’ve exported the workflow to a different computer with more memory (32GB compared to 8GB) and I’ve verified that using the same dataset, the error “Failed to initialize Python gateway” doesn’t occur. I’ve successfully repeated the execution twice

[steffen_KNIME]

Hi Francesco @falaimo,

good to know that memory solves the issue, thanks for sharing!
And sorry that we could not find any other issue for now.

Let us know if it occurs again!

Best regards
Steffen