Python Extension not recognizing Anaconda environment in Knime 3.7

Thanks. I’ll give a clean installation a try and circle back. I did notice that option to include path during the Anaconda installation. I selected to keep it separate as per their recommendation.

1 Like

No luck with a fresh installation
and
No luck with a older version of Knime (3.5)
and
No luck with turning off my windows firewall.

Not sure where to go from here…

Sorry to hear that. Sounds strange. I can think of these things:

  • check if Python does run at all. Maybe with a simple script in Spyder or a Jupyter notebook just to make sure it does run at all
  • could you run “conda info” and see what the results are (maybe we get a clue from that)
  • does the Python package show up at all in the configuration and do you still have the message that numpy is not installed?
  • with “conda list” you might check which version of numpy and pandas is there
  • KNIME has two Python scripting extensions. I mostly use the one from KNIME but maybe you give the one from Max Planck institute a try and see if this works to narrow down the problem
  • I read the recommendation first to use “conda update conda” to update the conda and then “conda update anaconda”
  • can you say which operating system you use / which version of windows

conda info

conda list

image

One other thing. If I start installing Keras in my environment via


with “conda install -c anaconda keras”

I get an impressive list of what Anaconda wants to do on the system with Updates and Downgrades. So compatibility really is a thing here. Since I am not very experienced with package management my idea would be to let Anaconda handle the packages.

And the Github page states that Keras is compatible with Python up to 3.6. So there is no guarantee it will work with 3.7 (I would assume it should but maybe has not been tested)

Success (kind of!)

Having absolutely no luck with the latest versions of Anaconda and Knime, I decided to go back in time. I uninstalled everything then installed:

Anaconda3 4.2.0 - this was the last version packaged with Python 3.5
Knime 3.5.3

I then ran:
conda update conda
conda update anaconda
conda install -c anaconda keras

Everything looked to install okay. I see pandas, numpy and keras in when entering the commad ‘conda list’

Here is my ‘conda info’
image

after installing the latest python extension available in the Knime 3.5.3 installation, I set the path in the Knime Python preferences to my main anaconda python.exe

It worked!

Now, when I tried to load the example Keras cats and dogs workflow, I received a notice that I need to update to Knime 3.6 to access some required nodes. I proceeded with the update which updated all my python integration extensions. I then downloaded the appropriate Keras extensions.

Looks like the python extension is still recognized! I ran a simple python workflow and it worked. I still need to work through the KERAS workflow but things are looking good!

Thanks for all the input and help troubleshooting. The Knime community is great!

3 Likes

Sorry for all the trouble with the installation. We tried to address this problem also with some new documentation, which can be found here:

https://docs.knime.com/2018-12/python_installation_guide/index.html

This doc is written for KNIME 3.7. We’re happy to receive feedback in case it doesn’t work or is too complicated :slight_smile:

1 Like

Hi Christian,

Thanks for passing on this documentation. I seem to be encountering an error when trying to create the env from the .yml file.

Here is my .yml file which I store in the main anaconda directory (C:\Tools\anaconda3)
image

I then run the command:
conda env create -f py36_knime.yml

I receive the following error:
image

I am using Anaconda 5.3.1 and Knime 3.7 and have run:
conda update conda
conda update anaconda
Here is my install info:

Any idea what is happening?

What I see is that there is no blank between name: and the actual name, that might play a role.

What I see next is that the examples from Anaconda all contain entries with blanks (or tabs) in front of the entries. Since Python is notorious for assigning actual meaning to blanks to structure functions it might well be that it also carries meaning here.

I have not tried the installation guide from KNIME myself. I will report back once I do.

image

1 Like

I am not 100% happy with this installation guide. It gives you basic informations but other things do not seem to work out instantly

I started by setting up a new environment within my existing Anaconda system thereby using the profile file that was provided in the installation guide.

then I struggled with the various scrips and sudo requirements of activating the thing and I am not entirely sure what lead to success in the end on my Mac. I did not give the path of some .sh script but rather it looks like this:

Now it accepts Python commands but I had to install additional Python libraries like the popular scikit-learn via the Anaconda Navigator since it was not in the original .yml configuration file

2 Likes

Hi Raj,

Please note that the “Can’t process without a name” error also occurs if the .yml file can’t be found. Can you try to use the absolute file location (i.e., conda env create -f C:/Tools/anaconda3/py36_knime.yml)?

Marcel

You are right, there must be a blank between “name:” and the actual name. As far as I can see, the file we provide on our documentation site does contain that blank. The missing blanks in front of the entries seem not to matter (at least they don’t on my Linux machine and conda 4.5.11).

Sorry to hear that the installation guide does not fully work for you. We appreciate your feedback :slight_smile:. As for the range of packages that are contained in the configuration file, we currently do not aim to provide a Python environment that contains all commonly used packages but rather the essential ones for the Python integration to function.

Would you mind to elaborate on that a bit, please? These steps are definitely something we need to make as clear as possible in the documentation. E.g., do you remember where you needed sudo privileges? As far as I can tell, these should not be necessary (except for installing conda in the first place, maybe).

2 Likes

Yes it does, I used it to create the environment via Anaconda Navigator.

Hi All,

Thanks for this continued discussion! I have not had the chance to try the potential fixes as I am away from my workstation. I will be back in the New Year and circle back with you on progress!

r

1 Like

Hi @mlauber71,

I’m currently working on improving the Python Installation Guide and your feedback would be really helpful. Could you maybe elaborate on the problems with the various scrips you were experiencing?

@DaveK I can try to help with my experiences although I am still in a process of understanding how to best handle the Python environments. If I try to follow the current guide on my Mac I always get error messages with the PATH part.

It is possible I somehow activated the environment by default. Or it might be just enough to add the path to the environment instead of the sh script - not sure yet.

Then I found you did not include the conda (and anaconda?) package in the yml configuration file. This might be added via the Anaconda Navigator later so one would be able to easily manage the packages via command line. Only if a very specific constallation of Python packages should be necessary to keep would it make sense to limit this function. But I am not sure yet.

You could send me an email and we might schedule a call.

1 Like

Hi mlauber71,

Could you maybe post the error message you are getting and the script you are using? Maybe you forgot to make the script executable?

Currently, just pointing to the environment also works. However, this is not our recommended way as this may lead to unexpected errors.

Which package are you referring to here? Do you mean scikit-learn?

Hi DaveK,

I had success creating the conda environment using the documentation. You were correct in pointing out that one should use the absolute location of the .yml file when creating the conda (conda env create -f C:/Tools/anaconda3/py36_knime.yml). It would be good to mention this in the installation documentation.

I am now at the stage of configuring KNIME. I created the .bat file as follows:

@REM Adapt the folder in the PATH to your system
@SET PATH=C:\Tools\anaconda3\Scripts;%PATH%
@CALL activate py36_knime || ECHO Activating python environment failed
@python %*

When I directly run this .bat file outside of KNIME is launches my python env as excpected:
image

however, when linking to this .bat in KNIME, I receive the following error:

I am running Knime 3.7 and I have updated to latest conda.

Any help would be much appreciated.

Hi rjesud,

I’m sorry that you have troubles with the Python installation. Could you maybe attach the KNIME log file in your next post? Also, it may be a permission problem. You could either try to start KNIME in administrator mode or move the .bat file to e.g. your desktop or user folder.

If this does not resolve the issue, you could provide us with additional information by following these steps:

  1. Download this archive PythonKernelTester.zip (2.6 KB). It contains a .py test script. Put it into the same folder as your .bat file.
  1. Execute the following command in the folder where you put both files:
    py36_knime.bat PythonKernelTester.py 3.0.0

Then paste the outputs here. That would be really helpful.

Cheers
Dave

3 Likes

Hi Dave,

When executing the command, I receive the following error:

image

To summarize

  1. I can link directly to the base python:
    image

  2. I cannot link to the .bat file (I tried running knime as admin and also placing the .bat on desktop)
    image

  3. I cannot link directly to the python.exe in the env folder:
    image

  4. I can execute the .bat directly (outside of knime) and it launches:

Thank you for your support with this issue!

sorry to hear that it is still not running. I still think you would have to make sure you have a ‘clean’ Anaconda managed Python installation. You could try:

  • reinstall a py36_knime environment via the Anaconda navigator like I showed in the screenshots above
  • update the py36_knime environment explicitly using the .yml file from the official installation guide *1) (https://docs.knime.com/2018-12/python_installation_guide/index.html)
  • always test the script “PythonKernelTester.py” mentioned above
  • maybe install Spyder in the py36_knime environment and try a few small commands to see if the Python environment does work at all
  • try to activate the py36_knime environment via Anaconda Navigator and see if that makes any difference
  • I am not sure about the .bat file if that is necessary and working, try both. .bat file and Python.exe directly
  • try installing additional packages strictly via Anaconda

I fear you have to do some trial and error to come up with a solution that does work.


*1)
source activate py36_knime
conda env update -f=/Users/m_lauber/Downloads/py36_knime.yml