Python Script in Knime Resource Usage

extensions
python
#1

Hi everybody,

I have Knime 3.7.2 version. Recently I’m using python script node in a knime workflow, and I noticed that every time I open this node or run it, my computer become slower. When I opened the task manager, it showed that there are many python applications running and consuming all the CPU resources, so always after I use this node, I need to end the task(python application) in order to continue using my computer normally. It seems that knime open a python app under every time whereas you open o run python script node. Is there any way to close or exit python, for every time a use this node without my intervention (after using it)?

I’d really appreciate your help!

Background information:
I am using python/knime integration via Anaconda.
Python Script (1=1)

0 Likes

#2

Hi Ultraman,

KNIME should automatically close the external Python process after each node execution. So this seems to be an error on our side. Would you mind uploading your knime.log file? Also, which operating system are you using?

Marcel

0 Likes

#3

Hi @MarcelW,

Thanks for your answer. I am using Windows 10. Here it is the knime log of the running process:

Tell me if you need any information.
Hope we can find a solution.

0 Likes

#4

Hi Ultraman,

The highlighted line in the screenshot you posted suggests that KNIME tried to forcibly close the external Python command after normal shutdown failed, which, as it seems, also failed.

On Windows, KNIME uses the taskkill command to do that. As a first step, I’d suggest to execute this command manually and see if it works this way. Here’s what you’d need to do:

  1. Run a Python node in KNIME.
  2. Open Task Manager, switch to Details tab, and locate the python.exe.
  3. Note down its process id (PID). (You may need to add the PID column to the Task Manager’s Details view first.)
  4. Open Windows Command Prompt (cmd). You can do this within Task Manager via File > Run new task > (type in:) cmd > OK. Make sure you create the task without administrator privileges (for now).
  5. Execute the following command: taskkill /f /pid <process-id-of-python-process> in Windows Command Prompt, where the id is the one from step 3 above. Please make sure that the id is correct, otherwise you may close the wrong process which could result in data loss or have other unwanted effects.
  6. See if the command ran successfully and the python.exe disappeared in Task Manager.
  7. If executing the command failed, retry from step 4, this time with administrator privileges.

Additionally, could you please upload your entire knime.log file here? There might be some more information buried elsewhere in the log, not visible in the screenshot. Thanks!

Marcel

0 Likes

#5

Hi @MarcelW,

Thanks for your response. I followed your instructions as you said, and when I executed step 7, it ended the python task. So I tried to run KNIME as an administrator, and it worked perfectly without the problem I mentioned before. It seems that the main issue is about administrator privileges. Running KNIME as an administrator solves the problem.

I think that there is not need to review the knime.log file.

Thanks again for your help!

3 Likes

#6

Hi Ultraman,

Glad to hear that it worked! We will still have a look internally at the underlying problem (the “Python messaging system could not be shut down gracefully. Process will be killed.” bit) since having to resort to taskkill is not how Python is supposed to be closed, especially not if that requires administrator privileges.

We’ll update this forum post once we made any progress in that direction.

Marcel

2 Likes