Python Extension not recognizing Anaconda environment in Knime 3.7

[mlauber71]

I am not 100% happy with this installation guide. It gives you basic informations but other things do not seem to work out instantly

I started by setting up a new environment within my existing Anaconda system thereby using the profile file that was provided in the installation guide.

then I struggled with the various scrips and sudo requirements of activating the thing and I am not entirely sure what lead to success in the end on my Mac. I did not give the path of some .sh script but rather it looks like this:

Now it accepts Python commands but I had to install additional Python libraries like the popular scikit-learn via the Anaconda Navigator since it was not in the original .yml configuration file

[MarcelW]

Hi Raj,

Please note that the “Can’t process without a name” error also occurs if the .yml file can’t be found. Can you try to use the absolute file location (i.e., conda env create -f C:/Tools/anaconda3/py36_knime.yml)?

Marcel

[MarcelW]

@mlauber71

What I see is that there is no blank between name: and the actual name, that might play a role.

You are right, there must be a blank between “name:” and the actual name. As far as I can see, the file we provide on our documentation site does contain that blank. The missing blanks in front of the entries seem not to matter (at least they don’t on my Linux machine and conda 4.5.11).

@mlauber71

Now it accepts Python commands but I had to install additional Python libraries like the popular scikit-learn via the Anaconda Navigator since it was not in the original .yml configuration file

Sorry to hear that the installation guide does not fully work for you. We appreciate your feedback . As for the range of packages that are contained in the configuration file, we currently do not aim to provide a Python environment that contains all commonly used packages but rather the essential ones for the Python integration to function.

@mlauber71

then I struggled with the various scrips and sudo requirements of activating the thing and I am not entirely sure what lead to success in the end on my Mac.

Would you mind to elaborate on that a bit, please? These steps are definitely something we need to make as clear as possible in the documentation. E.g., do you remember where you needed sudo privileges? As far as I can tell, these should not be necessary (except for installing conda in the first place, maybe).

[mlauber71]

@MarcelW

As far as I can see, the file we provide on our documentation site does contain that blank.

Yes it does, I used it to create the environment via Anaconda Navigator.

[rjesud]

Hi All,

Thanks for this continued discussion! I have not had the chance to try the potential fixes as I am away from my workstation. I will be back in the New Year and circle back with you on progress!

r

[DaveK]

Hi @mlauber71,

I’m currently working on improving the Python Installation Guide and your feedback would be really helpful. Could you maybe elaborate on the problems with the various scrips you were experiencing?

[mlauber71]

@DaveK I can try to help with my experiences although I am still in a process of understanding how to best handle the Python environments. If I try to follow the current guide on my Mac I always get error messages with the PATH part.

It is possible I somehow activated the environment by default. Or it might be just enough to add the path to the environment instead of the sh script - not sure yet.

Then I found you did not include the conda (and anaconda?) package in the yml configuration file. This might be added via the Anaconda Navigator later so one would be able to easily manage the packages via command line. Only if a very specific constallation of Python packages should be necessary to keep would it make sense to limit this function. But I am not sure yet.

You could send me an email and we might schedule a call.

[DaveK]

Hi mlauber71,

@mlauber71

If I try to follow the current guide on my Mac I always get error messages with the PATH part.

Could you maybe post the error message you are getting and the script you are using? Maybe you forgot to make the script executable?

@mlauber71

Or it might be just enough to add the path to the environment instead of the sh script - not sure yet.

Currently, just pointing to the environment also works. However, this is not our recommended way as this may lead to unexpected errors.

@mlauber71

Then I found you did not include the conda (and anaconda?) package in the yml configuration file.

Which package are you referring to here? Do you mean scikit-learn?

[rjesud]

Hi DaveK,

I had success creating the conda environment using the documentation. You were correct in pointing out that one should use the absolute location of the .yml file when creating the conda (conda env create -f C:/Tools/anaconda3/py36_knime.yml). It would be good to mention this in the installation documentation.

I am now at the stage of configuring KNIME. I created the .bat file as follows:

@REM Adapt the folder in the PATH to your system
@SET PATH=C:\Tools\anaconda3\Scripts;%PATH%
@CALL activate py36_knime || ECHO Activating python environment failed
@python %*

When I directly run this .bat file outside of KNIME is launches my python env as excpected:

however, when linking to this .bat in KNIME, I receive the following error:

I am running Knime 3.7 and I have updated to latest conda.

Any help would be much appreciated.

[DaveK]

Hi rjesud,

I’m sorry that you have troubles with the Python installation. Could you maybe attach the KNIME log file in your next post? Also, it may be a permission problem. You could either try to start KNIME in administrator mode or move the .bat file to e.g. your desktop or user folder.

If this does not resolve the issue, you could provide us with additional information by following these steps:

1. Download this archive PythonKernelTester.zip (2.6 KB). It contains a .py test script. Put it into the same folder as your .bat file.

2. Execute the following command in the folder where you put both files:
   py36_knime.bat PythonKernelTester.py 3.0.0

Then paste the outputs here. That would be really helpful.

Cheers
Dave

[rjesud]

Hi Dave,

When executing the command, I receive the following error:

To summarize

1. I can link directly to the base python:
   

2. I cannot link to the .bat file (I tried running knime as admin and also placing the .bat on desktop)
   

3. I cannot link directly to the python.exe in the env folder:
   

4. I can execute the .bat directly (outside of knime) and it launches:
   

   

Thank you for your support with this issue!

[mlauber71]

sorry to hear that it is still not running. I still think you would have to make sure you have a ‘clean’ Anaconda managed Python installation. You could try:

• reinstall a py36_knime environment via the Anaconda navigator like I showed in the screenshots above
• update the py36_knime environment explicitly using the .yml file from the official installation guide *1) (https://docs.knime.com/2018-12/python_installation_guide/index.html)
• always test the script “PythonKernelTester.py” mentioned above
• maybe install Spyder in the py36_knime environment and try a few small commands to see if the Python environment does work at all
• try to activate the py36_knime environment via Anaconda Navigator and see if that makes any difference
• I am not sure about the .bat file if that is necessary and working, try both. .bat file and Python.exe directly
• try installing additional packages strictly via Anaconda

I fear you have to do some trial and error to come up with a solution that does work.

---

*1)
source activate py36_knime
conda env update -f=/Users/m_lauber/Downloads/py36_knime.yml

[DaveK]

Hi rjesud,

there may be a .dll conflict on your system. The following command will show you the location of the .dll mentioned in the error message:
where mkl_intel_thread.dll
If it finds something under C:\Windows\System32\... you could try to remove that temporarily and see if Python works again.

Cheers
Dave

[genchak]

THank you @mlauber71 this one solved my problem. Now, I have openpyxl is working.

[asenkron]

log.txt (622.1 KB)

I had previously Anaconda installed, to achieve integration I created new environment with yml file shared in guide. I have followed and finished KNIME integration as mentioned in Guide. When I try to run workflow with Python nodes I receive several errors as below:

Here you can see that Python executable path seems OK.

Below the command window error:
ERROR Python Script (DB) 5:12 Execute failed: Could not start Python kernel. Error during Python installation test: Could not find python executable at the given location: python… See log for details.

I am attaching log file. Do you have any ideas what is wrong?

[MarcelW]

Hi asenkron,

Would you mind also attaching the knime.log file? (The one you attached is the eclipse .log file.)
Please see here (third and fourth paragraph) to find out how to locate the knime.log file.

Marcel

[asenkron]

Hi @MarcelW,

Sorry for delay, here I am attaching Knime.log file.

knime.log (2.5 MB)

[MarcelW]

Hi asenkron,

Thanks!

This is the root of the first error (first screenshot, Python Script (1 -> 1) node):
ModuleNotFoundError: No module named 'sklearn'.
This error can be resolved by installing scikit-learn in your local Python environment. I wonder, however, why this doesn’t properly show up as a user-visible node error.

Unfortunately, the log doesn’t seem to contain anything related to the failing Python Script (DB) node (second screenshot). Updating KNIME to version 3.7.2 might help here. There was a known bug in versions older than that, where Python versions 2 & 3 might get confused in some situations.

Marcel

[asenkron]

After turning back several months later, I reinstalled Anaconda environment, maybe updated on servers as well, now I am running without any error.

Thanks for response.