While running a workflow on a distant server, an error occured :
py4j.protocol.Py4JError: An error occurred while calling GATEWAY_SERVER.getCallbackClient. Trace:
Object ID unknown
ERROR KNIME-Worker-10-Python Script 3:1256 Node Execute failed: An exception occured while running the Python kernel. See log for details.
org.knime.python2.kernel.PythonIOException: An exception occured while running the Python kernel. See log for details.
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.net.ConnectException: Could not connect to the Python process.
… 6 more
May 12, 2023 4:07:31 PM org.apache.cxf.bus.osgi.CXFExtensionBundleListener unregister
I am not sure how to solve this with KNIME. I was thinking to assign more RAM, but I do not know if it could solve the problem. However, it does not seem that my workflow has a problem about python code I am using and so on. I am doing some parameters optimization with this workflow, so it takes a long time to complete the execution. I never saw this kind of error when it runs on my own laptop.
Could you please say the version of the KNIME AP you are using and confirm if you are using the most recent version of the Python Script node? Also, can you share the KNIME log file and either the script or a sample workflow? This will help us further investigate the issue.
I forgot to tell that I get this error when I execute it with command line. However, it do not seem that I get this error when I execute the wokflow with the graphical interface. But I have to do it by command line.
The error do not occur directly after executing workflows. The error do not seem to appear at a specific moment. It can takes few minutes like 1 hour before the error occur. It seem to be random.
I think the error seem to appear at “python script” nodes.
Since I do not have access to the input files, I am not able to investigate further, can you share a dummy workflow with some data which is not sensitive? Also, can you share the KNIME log file at the time of the error?
I noticed you are using the brute force strategy for optimization, I tried running the workflow in ‘bayesian optimization’ strategy, there was no issue until one of the python scripts (embed = nbook.get_umap_dimensions(df, nu, lmbda, y_axis_type)) failed with the below issue :
TypeError: Cannot use scipy.linalg.eigh for sparse A with k >= N. Use scipy.linalg.eigh(A.toarray()) or reduce k.
Very sorry for the delay. So, I found a solution to solve my problem. It is not the most clever one, but it was enough to do what I wanted. Since my error occurs in a random way from python nodes, I make some try catch loops. Thus, if the error occurs, then we repeat from last iteration starting from try node. Usually, it works the next time. I’ve been inspired by some sources that helped me: