Hello everybody. I am kinda new to the subject of environments. Can someone give me a help with integrating Python with Knime please?
I have installed Knime, Anaconda and Python.
I am trying to run this workflow
and I get these errors
ERROR Keras Network Learner 3:148 Execute failed: An error occurred while creating the Keras network from its layer specifications. Details: Deep learning Python back end installation tests failed. Cause: Python library ‘tensorflow’ or one of its dependencies is missing. Python library ‘keras’ or one of its dependencies is missing. In case Python packages are missing: you can create a new Python Conda environment that contains all packages required by the KNIME deep learning integrations in the “KNIME Deep Learning” Preferences. ERROR Keras Network Learner 3:148 Execute failed: An error occurred while creating the Keras network from its layer specifications. Details: Deep learning Python back end installation tests failed. Cause: Python library ‘tensorflow’ or one of its dependencies is missing. Python library ‘keras’ or one of its dependencies is missing. In case Python packages are missing: you can create a new Python Conda environment that contains all packages required by the KNIME deep learning integrations in the “KNIME Deep Learning” Preferences.
When installing python, I installed newest version 3.11.2 and after that in command prompt I pip installed tensorflow and keras.
When I go to knime preferences and go to Python settings it asks me to select ( or create new) Python 3 Conda environment. But all it offers as a base are Python 3.6, 3.7, 3.8, and 3.9. The new one I installed (3.11.2) doesnt get offered.
And under preferences under PYthon Deep learning it says Python version 3.9.13 and under that Library keras is missing and Library tensorflow is missing.
I am kinda confused about what the environment is and how to set it up. I assume that is a problem here.
Seems like I have Pythons installed all over. I have stand alone installation, then some of the versions got istalled with Anaconda and some got installed with Knime. And either there is a conflict or all apps should be using the same version? Please help with advice. Best regards, Dario
@Dalmatino16 setting up KNIME and Deep Learning always is a challenge. I have tried to write up this - also how to debug your setup. Along with some sample workflows that try to provide conda propagations to automatically install compatible versions for Keras and TensorfFlow. It still might need som effort:
@Dalmatino16 well in theory it should be as easy as just setting up a new Python deep learning environment from within the KNIME settings. But depending on your version and operating system it can be more complicated than that unfortunately.
@Dalmatino16 yes this is why I wrote the article. You can extract the necessary packages from the KNIME YAML configurations and then try to set them up with Miniforge/Conda. You have to use the specific restrictions for the Keras and TensorFlow versions to match the expected versions for the KNIME nodes - and sometimes a little back and forth is necessary. There is currently no consistent and easy-to-use setup for these deep learning packages, which might be due to them constantly evolving. Sorry not to be able to bring better news.
Hi again. Thans for suggestion. I tried to use yamal workflow but I got another issue there.First I got this:
Execute failed: OpenSSL appears to be unavailable on this machine. OpenSSL is required to
download and install packages.
After that I did install open SSL and then I got another problem:
WARN Conda Environment Propagation 3:32:0:104 Creating the environment failed. If an incomplete environment has been created, it will be removed and the original environment, if any, will be recovered.
ERROR Conda Environment Propagation 3:32:0:104 Execute failed: Encountered an SSL error. Most likely a certificate verification issue.
Do you have idea how to fix that? It happened when trying to run this node:
Do you have a working conda on your machine? If the propagation does not work you might have to resolve to installing it directly from a YAML file. You actually can extract the latest ‘official’ versions from the KNIME installation:
The YAML files for windows would then look like this. I have not yet figured out why the installation from within KNIME would sometimes fail. In every case it makes sense to have the latest version of conda installed.
The examples here use Python 3.6 or 3.7 - with the DL nodes it often is safer to have relatively older versions. I have postponed testing this on my Mac because there are several issues with the Apple Silicon implementation (older packages are often not available).
When using older versions they are sometimes not available on the channels so you might have to revert to versions near that like being described in the article.
# for Keras nodes and Python 3.7 you might also want to try the 3.6 version
name: py37_knime_dl_cpu # Name of the created environment
channels: # Repositories to search for packages
# - defaults # you can add this channel if your are OK with the licenses
# - anaconda # you can add this channel if your are OK with the licenses
- conda-forge
dependencies: # List of packages that should be installed
- python=3.7 # Python
- py4j # used for KNIME <-> Python communication
- nomkl # Prevents the use of Intel's MKL
- pandas # Table data structures
- jedi<=0.17.2 # Python script autocompletion
- python-dateutil # Date and Time utilities
- numpy # N-dimensional arrays
- cairo # SVG support
- pillow # Image inputs/outputs
- matplotlib # Plotting
- pyarrow=6.0 # Arrow serialization
- IPython # Notebook support
- nbformat # Notebook support
- scipy # Notebook support
- jpype1 # Databases
- tensorflow-mkl=1.15
- tensorflow-estimator=1.15 # or mac will install 2.6 which doesn't work
- tensorflow-hub
- keras=2.2.4
- h5py<3.0 # must be < 3.0 because they changed whether str or byte is returned
- libiconv # MDF Reader node
- asammdf=5.19.14 # MDF Reader node
- pip
- pip:
- onnx==1.5
- onnx-tf==1.5
# for TensorFlow 2 nodes and Python 3.6:
name: py36_knime_tf2_cpu # Name of the created environment
channels: # Repositories to search for packages
# - defaults # you can add this channel if your are OK with the licenses
# - anaconda # you can add this channel if your are OK with the licenses
- conda-forge
dependencies: # List of packages that should be installed
- python=3.6 # Python
- py4j # Used for communication with KNIME
- pandas=0.23 # Table data structures
- jedi=0.13 # Python script autocompletion
- python-dateutil=2.7 # Date and Time utilities
- numpy=1.16.1 # N-dimensional arrays
- cairo=1.14 # SVG support
- pillow=5.3 # Image inputs/outputs
- matplotlib=3.0 # Plotting
- pyarrow>=5.0.0 # Arrow serialization
- IPython=7.1 # Notebook support
- nbformat=4.4 # Notebook support
- scipy=1.1 # Notebook support
- jpype1=0.6.3 # Databases
- pip
- pip:
- tensorflow==2.2
- tensorflow-hub==0.8
Hi Steffen, nice of you trying to help. The thing is that I am new to this environment stuff, so not sure I am doing things right, and on the top of that I try work on a PC which has not been formatted for years and there might be some conflicts. For one thing I installed some things with pip install and some with conda install, and later I have read it is not recomended to mix. Anyway, at the moment I am using KNIME 4.7.1 and Anaconda Navigator 2.4.0. Will try what you suggested as soon as I gto PC and let you know. Best wishes, Dario
