Just curious to know how the execution of Python nodes are working.
Lets take the below scenario -
A workflow with Python node to fetch data from external source(it takes 5 mins to complete the process ) is on server. WF is executed in knime webportal, In backend what are the things will be happening?
If I close the tab and reopen the webportal again, I can see the jobs state (either executing or next screen), Need to know how executor / any process is executing the Python node.
KNIME just transfer the data from it’s node to external sources and get back the output or KNIME do some kind of changes in between?
If KNIME is just transferring the user entered/ provided data to external source, what is the advantage of KNIME in this case?
Any sort of information related to how KNIME nodes execution are happening will be really helpful to understand how it works in backend
I moved this to the server section as it fits better.
The KNIME Server consists of two different parts:
The TomEE or Tomcat part and
The Executor
The first part is among others managing the KNIME Server Users, the workflow repository, the webportal and the execution of the workflows (e.g. Scheduling).
The Executor is basically a KNIME Analytics Platform that runs in headless mode. The Executor is running all workflows.
So if you trigger a workflow to be executed on the KNIME Server either via WebPortal or KNIME Analytics Platform the Tomcat part will send a request to the Executor(s), if it is accepted by an Executor the workflow gets transferred to the Executor and is executed.
Basically the Python nodes will transfer the within the KNIME workflow calculated/processed data to Python including type conversions (if necessary), process them in Python and get it back to process it further within the KNIME workflow.
The advantage is to use Python scripting in addition to the possibilities of all KNIME nodes without leaving the KNIME environment.
Hope this helps you, I am happy to elaborate if there are any additional questions on your side open.