Hello. I seem to be having an issue when I use the “Keras Network Learner” node. I keep getting an error that says:
‘TF2Network’ object has no attribute ‘train’
Here is a entry in the log when the error occurred:
2022-01-05 15:42:50,827 : ERROR : KNIME-Worker-78-Keras Network Learner 7:16 : : DLKerasLearnerNodeModel : Keras Network Learner : 7:16 : ‘TF2Network’ object has no attribute ‘train’
2022-01-05 15:42:50,828 : ERROR : KNIME-Worker-78-Keras Network Learner 7:16 : : Node : Keras Network Learner : 7:16 : Execute failed: An error occured during training of the Keras deep learning network. See log for details.
java.lang.RuntimeException: An error occured during training of the Keras deep learning network. See log for details.
at org.knime.dl.keras.base.nodes.learner.DLKerasLearnerNodeModel.handleGeneralException(DLKerasLearnerNodeModel.java:739)
at org.knime.dl.keras.base.nodes.learner.DLKerasLearnerNodeModel.executeInternal(DLKerasLearnerNodeModel.java:714)
at org.knime.dl.keras.base.nodes.learner.DLKerasLearnerNodeModel.execute(DLKerasLearnerNodeModel.java:316)
at org.knime.core.node.NodeModel.executeModel(NodeModel.java:549)
at org.knime.core.node.Node.invokeFullyNodeModelExecute(Node.java:1267)
at org.knime.core.node.Node.execute(Node.java:1041)
at org.knime.core.node.workflow.NativeNodeContainer.performExecuteNode(NativeNodeContainer.java:559)
at org.knime.core.node.exec.LocalNodeExecutionJob.mainExecute(LocalNodeExecutionJob.java:95)
at org.knime.core.node.workflow.NodeExecutionJob.internalRun(NodeExecutionJob.java:201)
at org.knime.core.node.workflow.NodeExecutionJob.run(NodeExecutionJob.java:117)
at org.knime.core.util.ThreadUtils$RunnableWithContextImpl.runWithContext(ThreadUtils.java:365)
at org.knime.core.util.ThreadUtils$RunnableWithContext.run(ThreadUtils.java:219)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at org.knime.core.util.ThreadPool$MyFuture.run(ThreadPool.java:123)
at org.knime.core.util.ThreadPool$Worker.run(ThreadPool.java:246)
Caused by: org.knime.python2.kernel.PythonIOException: ‘TF2Network’ object has no attribute ‘train’
at org.knime.python2.kernel.messaging.AbstractTaskHandler.handleFailureMessage(AbstractTaskHandler.java:146)
at org.knime.python2.kernel.messaging.AbstractTaskHandler.handle(AbstractTaskHandler.java:92)
at org.knime.dl.python.core.DLPythonAbstractCommands$DLTrainingTask.runInternal(DLPythonAbstractCommands.java:925)
at org.knime.core.util.ThreadUtils$CallableWithContextImpl.callWithContext(ThreadUtils.java:381)
at org.knime.core.util.ThreadUtils$CallableWithContext.call(ThreadUtils.java:267)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
I would appreciate if anyone could help me solve this issue. Thank you!
happy to try to help you finding the issue. Are you allowed to share your workflow?
If not, can you please post a screenshot of the workflow to get some more information about how you define your network and what kind of network you are trying to train?
@akhtarameen were you able to test the suggestions in this post and the associated workflows using Keras and Conda environment propagation (for MacOSX and Windows)?
If this does work initially as @Kathrin has suggested it would be interesting to see your setup and especially the build-up of the network. If you could share that someone might be able to have a look.
This sounds like you’re trying to train a TensorFlow 2 model with the Keras Network Learner node, is that correct? Unfortunately, the Keras Network Learner node (and the entire KNIME Deep Learning - Keras Integration, for that matter) only supports TensorFlow 1. Currently, you’d need to resort to Python scripting via the DL Python Network Learner – KNIME Community Hub node if you wanted to train a TensorFlow 2 model.
Thank you everyone for the responses. Appreciate the help. Here is a screenshot of my workflow. It is actually just an example workflow that was available in KNIME hub and hence I will also post the link to that as well.
@MarcelW I am a little confused by what you mean. Could you please elaborate a little further if possible? Thank you.
@mlauber71 Do check the screenshot, where I tried using the conda environment propagation along with the keras nodes but it didn’t seem to work. Any ideas?
@akhtarameen I have adapted the example and added a propagation myself and successfully run it on a Windows machine and it seems to work (hope @Kathrin does not mind). The propagation would have to be pre-run in order to work (so best you use my example). You might have to allow it to create a new Python environment. I initially did find the propagation concept a little confusing to be honest.
Then you might want to include more information about your system and what the error message says; possibly after using the example. Please note the fully executed workflow gets too large for the hub or the forum.
And then there might be hope for all the Python version problems since KNIME has announced a cooperation (integration) with Anaconda. They are a little short on information about licenses (Anaconda did some changes there recently) but there seems to be some development.
KNIME supports TensorFlow 2 via the KNIME Deep Learning - TensorFlow 2 Integration (which should also work on Windows ). It’s just the Keras nodes that only support TensorFlow 1. The DL Python scripting nodes are a little special in that they are installed alongside both the Keras integration and the TensorFlow 2 integration and support both libraries.
Looking at your workflow, I share your confusion given that you’re only using Keras layer nodes to build the model. Could you please share the list of packages of:
The “Keras” conda environment configured in the Preferences (File > Preferences > KNIME > Python Deep Learning), and
The conda environment configured in the Conda Environment Propagation node (Node 190 in your screenshot, or the one in the modified workflow provided by @mlauber71, if that modified version didn’t fix your error)
with us? You can display the list e.g. by executing conda list -n <your-environment-name> via the terminal, or by using the Anaconda Navigator.
Well … I had problems installing an anaconda tensorflow version >=2.2. It would only install versions up to 2.1 in a configuration that was acceptabe to KNIME. But maybe you could provide an Environment Propagation that would work (preferably with miniconda and usage of conda-forge - where unfortunately the latest version of tensorflow forf windows is v1.14.0) … and working with this example:
With a tensorflow version 2.1 I get some strange error messages. That would be a thing for a new forum entry maybe.
I think what was going wrong was the versions of the different packages I had before like tensorflow, keras, etc were not matching well together. After I used the workflow provided by @mlauber71 everything seemed to run just fine.
I think, it all started when I couldn’t automatically create the deep learning environments for keras and tensorflow by going to KNIME → Preferences → Python Deep Learning. It kept giving me an error that said “Environment creation failed: The following specifications were found to be incompatible with each other”. Thus, I manually installed the required packages for the virtual environment which I kept using with the initial workflow I sent above.
The conda environement configured in my screenshot is the same as the Keras conda environment I manually configured. I have attached the list of packages I have installed for this environment below. Hope you can help me identify what the issue is.
We’re going to extend the options to auto-create TensorFlow 2 environments offered in the Python Deep Learning Preferences soon, such that they support both higher TF2 and higher Python versions (similar to what we already added in the Python Preferences last release, where you can now create Python 3.7-3.9 environments in addition to the old 3.6 ones). I hope that already helps a bit.
I’m not sure about the “conda-forge-usage”-option. I assume you’re referring to that because of the licensing. Miniconda itself should already work with KNIME, at least I use it all the time without problems so far.
KNIME requires TF 2.2 or higher to work (at least “officially” in that we don’t test older versions, no idea if they could work). KNIME should actually reject older versions with an error, isn’t that the case?
suggest that you were indeed trying to use the Keras nodes with TensorFlow 2, which does not work (as mentioned above, the Keras nodes only work with TensorFlow version 1). I do wonder, however, why KNIME didn’t complain about that when executing the layer nodes (“Keras Input Layer”, “Keras Embedding Layer”, etc.) but only failed in the Keras Network Learner. That’s seems to be a bug on our side, sorry for the confusion!
That’s weird. What version of KNIME Analytics Platform do you use? And which operating system?
You could try to repeat this step but specify the Keras and TensorFlow versions such that they correspond to the ones we recommend for use with the Keras nodes (TF 1.12.0 and Keras 2.2.4). E.g. like this: conda install tensorflow-gpu=1.12.0 keras=2.2.4
@MarcelW what would be great is if KNIME could provide ‘curated’ yml files in addition to the possibility to install an environment via the preferences. Since then one could try and make it work on a machine while having the basic settings ready (basic Python version). Another possibility would be to provide a Conda Environment.
If I tried to install Trensorflow 2 on a Windows machine in a ‘KNIME-accepted/supported version’ it would crash. And currently I and not able to do it, although I have not tried that hard.
Indeed I try to use Miniconda (which is fine and not bloated) with conda-forge to see if that is enough to be used with Deep Learning environments. I am looking forward to the planned KNIME and Anaconda integration. And indeed one reason is the licensing. While what I saw would not break the bank if you use it in a professional environment it might be good to have a ‘license free’ setting for tests and trainings within a company.
with Tensorflow 2.1 (just short of the requirement, damn) the node would run and then in the end crash with a strange message. I assume they have changed some settings between 2.1 and 2.2 and KNIME would expect the later. OK maybe we stop here and I take a deeper look another time
That makes sense. We’re currently working on something which should address having “the basic settings ready” quite nicely and hope to have it ready soon, let’s see .
As for the curated YML files: we don’t really promote using them publicly, but the files that underlie the Preferences’ auto-creation functionality are available with the source code here. Perhaps these files can serve as basis when trying to get an environment to work on a machine.
). It’s just the Keras nodes that only support TensorFlow 1. The DL Python scripting nodes are a little special in that they are installed alongside both the Keras integration and the TensorFlow 2 integration and support both libraries.