I have set up the Python and Deep Learning libraries as instructed. When I use the Python Script, I can run the model using Keras. However, it is failing when using Keras network learner. I have also tried to launch Knime using “-clean” mode but without success.
I tried using new environment also - It is not working
One of the reason that I figured out is that in the setting of Python Deep learning, it shows “No environment” when I open it…after few minutes only my conda environment gets loaded. is it possible that Keras doesn’t find any enviorment to work upon?
Just to make sure, when you click on the New environment… button on the Python Deep Learning configuration page, you are asked to provide a name for your new environment (by default, I think it’s “py3_knime_dl”) and choose whether you want a CPU or GPU basis.
Did you do this? It takes a few minutes to build the environment, but afterward you should be able to select it from the dropdown. In the screenshot above, you still have no environment selected, so the Keras Network Learner node won’t be able to find it.
Or perhaps you did build the new environment, but for some reason you can’t select anything from the dropdown at all? That would be quite strange.
If you have a little more detail, that would help. Thanks!
I have also built the environment using New environment. However, after I select the enviornment from the dropdown, press “Apply and close” (or apply), the environment resets to “No Environment” and hence Keras couldn’t find it. It happens every time.
I have created a small screen recorder to show the problem. It is a three minute video but you can skip the part where Keras node is trying to copy the input at port 1.[Two instances - first from 0:05 to 0:46 and another from 1:50 to 3:14)
Setting up deep learning environment with KNIME can sometimes be tricky. You could check these steps and make sure you have the environment running from within python (and also KNIME). Also at the end of the thread there are some hints by @aworker that might or might not be relevant for your case.
And yes I am aware that repeating a task time and time again hoping for a different outcome is sometimes regarded as one sign of madness …
The Keras Network Learner node performs an integrity check on the Python environment it is supposed to use, before executing the actual model training. This is done to make sure that the required libraries (Keras, TensorFlow, etc.) are properly installed and their versions match the ones that KNIME supports.
This check times out after a certain duration to prevent the node from freezing in case of unexpected external circumstances (e.g., the Python process crashes).
The part “Installation test for Python back end […] timed out” in the error message you experience suggests that such timeout is exactly what happens in your case.
Given that, on the Python Deep Learning preference page in your recording, the steps “Testing Conda installation…”, “Collecting available environments…”, and “Testing Python environment…” take a very long time to complete, I would suspect that the reason for the timeout is that executing conda commands is slow on your machine. This is a known problem on some Windows machines (and presumably for specific Conda versions only) that we have also experienced on some of our internally used machines.
Here is what you can do:
First, try to check whether the problem really is on the Conda side.
Open an Anaconda Prompt window. (You should be able to find Anaconda Prompt via the Windows Search/Start Menu if you are unsure where it is located.)
Type in and run: conda activate py3_knime_dl
Wait for the above command to terminate. If this took a significant amount of time (multiple seconds or more) or the command behaved otherwise unexpectedly, it could be the reason why you experience these timeouts. If not, please let us know, then we will have to look for other possible error sources.
If it is indeed Conda being slow:
Add the option -Dknime.dl.installationtesttimeout=3600000 to your knime.ini file (on a new line at the very bottom of the file). You can find the knime.ini in your KNIME installation directory. The option increases the timeout of the integrity check to one hour (3,600,000 milliseconds).
Try to rerun your workflow. The Keras learner node will now be stuck at “Copying input object at port 1” even longer (up to an hour, see above) since that is when the check is carried out.
If, after that, the node continues to execute, we have successfully worked around the problem. If not, again, please let us know.
The above is of course just a workaround and waiting for the node to execute for minutes is not ideal. So furthermore you could try to:
Temporarily deactivate all antivirus software on your computer (including Windows Defender) and see if this improves things. You do this at your own risk, however. This is not a permanent solution but should only be done once to gain more insight into the problem. If it helped, you may want to consider adding exceptions for Conda to the antivirus programs’ filters. Again, at your own risk.
Downgrade Conda to version 4.7.12. In our experience, this version turned out to work well on some Windows machines on which we previously experienced the problem at hand.
Thanks for looking into the problem.
I don’t think Conda is slow because py3_knime_dl gets activated very fast on
I can’t tamper with the antivirus as it is IT controlled so I will try to downgrade Conda to 4.7.12 and will see if it improves.
By adding the option to Knime.ini file, my problem was solved ( I didn’t do it for 1 hours but for only 5 mins) . I am providing a video for your reference to look at the speed of the execution for other suggestions.
Please let me know if there is anything else that I need to try.
Happy to hear that at least you’re now able to execute the workflow .
The speed of the command executions in your video indeed seems a bit slow (on my Linux machine, they really complete instantaneously), but not too bad. I’ll double check at our end to see if there’s something else that could further prolongate the delay.
I believe there’s nothing more you could do right now, unfortunately. I hope the current delay when executing a deep learning node is tolerable.