So first of all: let me assure you that we are aware of the performance issues of our Python nodes and are actively working on improving them . At the moment, the main problem is that all of the input data to a Python node needs to be copied and transferred from KNIME to the external Python process. Likewise, all of the output data needs to be transferred from Python back to KNIME. So roughly speaking, the input data is read twice (first in KNIME, then in Python – and copied once in between) and the output data is written twice (first in Python, then in KNIME – and, again, copied once in between). This overhead will be reduced in the future, but unfortunately, I cannot make any predictions as to when these improvements will be available.
In the meantime, you could try to change the library that is used for copying and transferring the data, to Apache Arrow via File > Preferences > KNIME > Python > Serialization library. Note that you may need to install an additional package – pyarrow, version 0.11 – in your Python environment for this to work. If your Python environment is a Conda environment that has been created via KNIME, then this package will already be installed.
But as mentioned above, the underlying problem of having to copy and transfer the data still persists. So changing the serializer will only improve things up to some point.
This is a similar issue. Here, the entire input data is first transferred from KNIME to Python and only after this is done, the script begins to execute and transfer the data to the GPU. What KNIME would need to do here, instead, is to stream the individual training batches from KNIME via Python to the GPU. Note that this is exactly what the Keras Network Learner node does. So if your training routine does not require any custom scripting, you could also try to use that node.