I installed the latest version of Knime (3.7.0 - with python extension 3.7) and Anaconda ( 5.3.1- with python 3.7). From the anaconda cmd prompt, I create a new environment using the command:
conda create -y -n py35_knime python=3.5 numpy jedi pandas keras
Please note, I create a Python 3.5 environment because Anaconda will not let me install keras with Python 3.7. In anaconda i get the error:
UnsatisfiableError: The following specifications were found to be in conflict:
-keras -> python[version=’>=3.5,<3.6.0a0’]
The python 3.5 environment is created just fine. I can see the list of installed components in the anaconda cmd (numpy=1.15.4, pandas=0.23.4, keras=2.2.4). It resides here :“C:\Tools\Anaconda3\envs\py35_knime”
I then create a .bat file with script: @REM Adapt the directory in the PATH to your system @SET PATH=C:\Tools\Anaconda3\Scripts;%PATH% @CALL activate py35_knime || ECHO Activating py35_knime failed @python %*
Now here’s the problem:
In Knime, when I go to File>preferences>python and set the Python 3 to my .bat file, I get an error which reads “Python Installation could not be determined”
So I tried to troubleshoot
If I execute the .bat file outside of Knime, it launches Python 3.5 just fine.
In Knime Python preferences, if I set the path directly to the Python .exe in the main Anaconda folder, it recognizes Python 3.7 but says numpy and pandas is missing
If I create a Python 3.7 environment from Anaconda including numpy and pandas components, I can link to the python.exe in the new envs folder from Knime preference but the error for missing numpy and panda persists
If I change my .bat file to activate the base anaconda python install, the link this path in knime, I get the error of “Python Installation could not be determined”. Again executing the .bat outside of knime launches the expected version.
I’m at a loss. Can someone help me get Python, Numpy, Pandas, and Keras installed and linked to Knime 3.7
If I set the path in knime python preferences directly to the python.exe in my 3.5 env folder, I get the error"
“Python Installation could not be determined”
I do not have that much experience with using different Python environments with Anaconda and KNIME but what I saw was that it is best to let Anaconda handle the package management and mostly use Python packages that are compatible with the current version.
conda install -c conda-forge keras
Once I made the mistake of updating single Python packages with the blue links in the environment, big mistake. I received a lot of error messages.
Since then I would only use conda update anaconda
to update my packages.
And in my Anaconda environment there is only Python 3.6.7
Sometimes when I install packages from outside anaconda or from some other repository I then let “conda update anaconda” sort it out again. And sometimes it suggests a small downgrade of a package in order to keep the versions in sync.
I ran conda clean --packages
and also
conda update anaconda
but I still get the error. So instead, I tried pip
pip install keras
pip install numpy
pip install pandas
everything looked to download and install correctly.But I still cannot get Knime to recognize the components. Are there specific versions I should have? Here is the detail when installing:
My only idea would be to completely reinstall anaconda and then install all packages necessary via a clean anaconda source. Also anaconda offers the possibility to include conda in the windows path statement but they suggest starting a prompt separately that recognises the executable and not promote the python exe across the whole system.
The downside is you cannot use conda from every prompt but would have to start it specifically from the anaconda group/folder.
Thanks. I’ll give a clean installation a try and circle back. I did notice that option to include path during the Anaconda installation. I selected to keep it separate as per their recommendation.
Sorry to hear that. Sounds strange. I can think of these things:
check if Python does run at all. Maybe with a simple script in Spyder or a Jupyter notebook just to make sure it does run at all
could you run “conda info” and see what the results are (maybe we get a clue from that)
does the Python package show up at all in the configuration and do you still have the message that numpy is not installed?
with “conda list” you might check which version of numpy and pandas is there
KNIME has two Python scripting extensions. I mostly use the one from KNIME but maybe you give the one from Max Planck institute a try and see if this works to narrow down the problem
I read the recommendation first to use “conda update conda” to update the conda and then “conda update anaconda”
can you say which operating system you use / which version of windows
One other thing. If I start installing Keras in my environment via
with “conda install -c anaconda keras”
I get an impressive list of what Anaconda wants to do on the system with Updates and Downgrades. So compatibility really is a thing here. Since I am not very experienced with package management my idea would be to let Anaconda handle the packages.
And the Github page states that Keras is compatible with Python up to 3.6. So there is no guarantee it will work with 3.7 (I would assume it should but maybe has not been tested)
Having absolutely no luck with the latest versions of Anaconda and Knime, I decided to go back in time. I uninstalled everything then installed:
Anaconda3 4.2.0 - this was the last version packaged with Python 3.5
Knime 3.5.3
I then ran:
conda update conda
conda update anaconda
conda install -c anaconda keras
Everything looked to install okay. I see pandas, numpy and keras in when entering the commad ‘conda list’
Here is my ‘conda info’
after installing the latest python extension available in the Knime 3.5.3 installation, I set the path in the Knime Python preferences to my main anaconda python.exe
Now, when I tried to load the example Keras cats and dogs workflow, I received a notice that I need to update to Knime 3.6 to access some required nodes. I proceeded with the update which updated all my python integration extensions. I then downloaded the appropriate Keras extensions.
Looks like the python extension is still recognized! I ran a simple python workflow and it worked. I still need to work through the KERAS workflow but things are looking good!
Thanks for all the input and help troubleshooting. The Knime community is great!
What I see is that there is no blank between name: and the actual name, that might play a role.
What I see next is that the examples from Anaconda all contain entries with blanks (or tabs) in front of the entries. Since Python is notorious for assigning actual meaning to blanks to structure functions it might well be that it also carries meaning here.
I have not tried the installation guide from KNIME myself. I will report back once I do.
I am not 100% happy with this installation guide. It gives you basic informations but other things do not seem to work out instantly
I started by setting up a new environment within my existing Anaconda system thereby using the profile file that was provided in the installation guide.
then I struggled with the various scrips and sudo requirements of activating the thing and I am not entirely sure what lead to success in the end on my Mac. I did not give the path of some .sh script but rather it looks like this:
Now it accepts Python commands but I had to install additional Python libraries like the popular scikit-learn via the Anaconda Navigator since it was not in the original .yml configuration file
Please note that the “Can’t process without a name” error also occurs if the .yml file can’t be found. Can you try to use the absolute file location (i.e., conda env create -f C:/Tools/anaconda3/py36_knime.yml)?
You are right, there must be a blank between “name:” and the actual name. As far as I can see, the file we provide on our documentation site does contain that blank. The missing blanks in front of the entries seem not to matter (at least they don’t on my Linux machine and conda 4.5.11).
Sorry to hear that the installation guide does not fully work for you. We appreciate your feedback . As for the range of packages that are contained in the configuration file, we currently do not aim to provide a Python environment that contains all commonly used packages but rather the essential ones for the Python integration to function.
Would you mind to elaborate on that a bit, please? These steps are definitely something we need to make as clear as possible in the documentation. E.g., do you remember where you needed sudo privileges? As far as I can tell, these should not be necessary (except for installing conda in the first place, maybe).
Thanks for this continued discussion! I have not had the chance to try the potential fixes as I am away from my workstation. I will be back in the New Year and circle back with you on progress!
I’m currently working on improving the Python Installation Guide and your feedback would be really helpful. Could you maybe elaborate on the problems with the various scrips you were experiencing?
@DaveK I can try to help with my experiences although I am still in a process of understanding how to best handle the Python environments. If I try to follow the current guide on my Mac I always get error messages with the PATH part.
It is possible I somehow activated the environment by default. Or it might be just enough to add the path to the environment instead of the sh script - not sure yet.
Then I found you did not include the conda (and anaconda?) package in the yml configuration file. This might be added via the Anaconda Navigator later so one would be able to easily manage the packages via command line. Only if a very specific constallation of Python packages should be necessary to keep would it make sense to limit this function. But I am not sure yet.
You could send me an email and we might schedule a call.
. As for the range of packages that are contained in the configuration file, we currently do not aim to provide a Python environment that contains all commonly used packages but rather the essential ones for the Python integration to function.