PythongSource node memory leaks

I wanted to post this here first, before reporting it as a bug. I'm not sure if I'm doing something wrong, or Knime. 

I'm on OSX 10.11.5. Knime 3.1.2

I'm new to Knime, but an advanced Python programmer. I'm experimenting with passing strings in from an external Python script, which are then passed into a 'PythonSource' Knime node for processing. Right now, it's just a 'Hello World' style script.

Everything is working correctly, and apprently finishing without error...however, after running my test script a few times, I notice a lot of Python kernels left hanging around. Closing Knime doesn't have any effect. I have to manually kill all these zombies. 

(See the attached images)

Everything (all python scropts and knime) seem to be running successfully and ending normally. 

Is there something additional I must be doing to properly end the PythonSource Knime script?

 

Hi,

the Python Kernel is shutdown as soon as it is not needed anymore (the dialog is closed or the node finished execution successfully or with an error).

The Python Kernel has three mechanisms for shuting down. The first is to send it a shutdown signal. This only works if the Kernel is still responsive (which should be the case if it did successfully execute and deliver the results). After it had some time to shutdown it will be forcefully killed (with 'kill -KILL <PID>') via PID (obtained in the Kernel via os.getpid()). After that we also call Process.destroy() from Java. In case you wrapped the Python start in a script this kills the Python Kernel first and then the script.

Did you start Python with a wrapper script (the path to python executable in the preference page points to a script)? Can you provide the full Python log (View -> Open KNIME log)?

Thanks, Patrick

Hi, were you able to find a solution to this. I’m facing the same problem with the “Python Script (1⇒1)” node even though the script executes successfully!

Hi @sharvilp,

could you give us a little more information? What KNIME and Anaconda version are you using? Can you described in more detail what you mean with “the same problem”? Are there Python processes that are not closed after all Python nodes finished running? Also, if possible could you maybe create a workflow showing the issue? Or/And a short step by step description how to reproduce the problem?

Thanks in advance.

Cheers
David

2 Likes

Please also see here (especially if you’re on Windows): Python Script in Knime Resource Usage

2 Likes