KNIME Python/Anaconda env configuration/assigning issues

Hi,

I’m having a problem getting Python to work in KNIME…

I’m following

  1. https://www.knime.com/blog/setting-up-the-knime-python-extension-revisited-for-python-30-and-20
  2. https://www.knime.com/blog/how-to-setup-the-python-extension

I’ve installed Anaconda 3, and can run both 2x & 3x versions of python in Jupyter notepad. I’m comfortable with creating environments with both the terminal CLI (as per 1 above) and via the GUI in Anaconda 3.

I’ve created the bash script (as per 1 above) and can call this from the terminal successfully, but when I try and add it (as per 1 above) to:

File -> Preferences -> KNIME -> Python -> Python 3 Path to Python 3 executable

I get the following error:

activate: no such file or directory

this looks like some sort of problem in KNIME executing the

source activate py35_knime

command which is in the .sh file, thinking activate is a directory rather than part of the source command.

has anyone come across this before? any workarounds or solutions? Have I missed something out of the set-up process?

I’m using KNIME 3.5.2 on Apple Mac0S High Sierra 10.13.3.

Any help much appreciated.

Cheers,

Andrew

Hey Andrew,

the activate command is provided in the bin folder of your anaconda directory.
Did you add the following line to your .sh script:
export PATH="PATH_WHERE_YOU_INSTALLED_ANACONDA/bin:$PATH"
If so, can you confirm that “PATH_WHERE_YOU_INSTALLED_ANACONDA/bin/activate” exists?
You may also add the the export command to your .bashrc file.

I hope that helps you fix your problem. Please do not hesitate to reach out to me again if the problem persists or if you have further questions.

Best,

Clemens

Just point to the python executable in anaconda3/bin/

I tried following the .sh with a new conda environment as well and it didn’t work for me either.
If I ran the .sh script by itself it started the environment without issues.

Hey @sunworshipper,

what exactly did you try? We couldn’t reproduce the problems locally, so maybe if you can provide some details (KNIME version, conda version, OS, error message), maybe we can try reproducing again and come up with an update for our setup guide.

Thanks!!

Christian

High Sierra 10.13.3
Python 3.6.4 |Anaconda custom (64-bit)
conda 4.4.11
KNIME 3.5.2

Note that I tried with Python3.6.

I followed these instructions:

https://www.knime.com/blog/setting-up-the-knime-python-extension-revisited-for-python-30-and-20

When I made my sh script I could execute it with sh knime_py36.sh, so I could run the script and that would start the knime environment I created in conda. That makes me think that the .sh script is setup correctly.

When pointing Knime to the .sh file it did not resolve. I got an error in line 8 of the sh script, just like Andrew.

That is this line:

source activate py36_knime

The workaround was to just point instead to the .sh to the python executable in anconda3/bin/
This essentially makes you work from the conda root environment, but I am ok with that.
Although it would be optimal if the instructions worked as advertised.

1 Like

Hello,

I tested it on our Mac system, which is also running High Sierra.
I set up the Anaconda environment with Python 3.6.4 and it works fine for me.
One problem I’ve encountered is that when creating the .sh file, that TextEdit changed my ’ " ’ as it started in Rich Text format. When i switched to Plain Text I had to replace ’ “ ’ and ’ ” ', which encapsulate the path to ’ " ’ (I hope you can see the difference in the forum).
Could you please post the error you’ve got, so we could further investigate this behaviour.
Another thing could be, that py36_knime isn’t the actual name of the environment due to a typo or so?

Cheers,
Moritz

@clemensvonschwerin Thanks for the message - yes I had the PATH environment configured correctly, I could see it via echo, and had added to my .bash_profile. This was all configured prior to the problem occuring.

@sunworshipper that’s the exact issue, and I’ve had to put the same work around in, point direct to the executable via preferences

@christian.dietz - the problem is in the execution of the source activate [env] command in the .sh, the script works when called from the terminal CLI, but doesn’t work when called from the preferences tab. It’s definitely the execution of the script as (as @sunworshipper has stated) you can point directly to the python executable in that [env] folder.

@moritz.heine I’m not I can add any more details - they [env] exists, the .sh works if I directly invoke it from the CLI, and the error is as above (activate: no such file or directory) referenced by the line in the .sh file. The error seems to be the execution of the .sh file and failing on the source activate [env] command

@Andrew_Bolton could you please send us your KNIME log file? We’ve tested it on one of our Mac systems and it works as intended. Even when not making it executable with the chmod command I obtain another error and can’t even run it from the terminal.
We will test it on further machines and the log file would help us to narrow down the error.
You can find the log file in your KNIME workspace under:
your_knime_workspace/.metadata/knime/knime.log

Alternatively, you can open the KNIME log in KNIME under View -> Open KNIME log and copy the content and paste it into an .txt file.

Thanks!
Moritz

@moritz.heine

if I set the Preferences to the .sh file the error is dynamically written to the preferences GUI in red, and the snippet from the log is;

2018-03-20 11:06:52,815 : DEBUG : Thread-15 : PythonKernelTester : : : Executed command: /Volumes/DataOps/ps35_knime.sh /Applications/KNIME 3.4.1.app/Contents/Eclipse/plugins/org.knime.python2_3.5.0.v201712011355/py/PythonKernelTester.py 3.0.0
PYTHONPATH=:
PATH=/usr/bin:/bin:/usr/sbin:/sbin
Error during execution: /Volumes/DataOps/ps35_knime.sh: line 4: activate: No such file or directory

but the path is set in the .sh script file

export PATH="/Volume/DataOps/05_Toolset/anaconda3/bin:$PATH"

it looks like KNIME isn’t setting either the PYTHONPATH or updating the PATH environment variable, and therefore the source activate command fails.

the file is set to 755, so should be executable by KNIME?

(as the anaconda path is also in my .bash_profile if I echo $PATH from the Terminal CLI I get /Volumes/DataOps/05_Toolset/anaconda3/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin and, as noted above, the .sh works as expected when invoked from the Terminal CLI)

If I set the preferences to the path of python executable in the preferences GUI, the GUI dynamically updates with the correct version of python, and I can’t see anything being written to the log.

Thanks for your reply.
I’ve seen that you updated KNIME from version 3.4.1 to 3.5.2.
Could you also check, which Python extension you have installed?
We have two Python Integrations

  • Python Integration 3.5.2
  • Python Integration (deprecated) 3.5.0

So it could be that when you’ve installed Python previously it only updated to the version 3.5.0, which seems to contain a bug concerning the Preference page where you can select your .sh file.
If that is the case you could install 3.5.2 additionally, which may fix the problem.

@moritz.heine Hi, I’ve got

  • Python Integration 3.5.2.v201802051356
  • Python Integration (deprecated) 3.5.0.v201712022346
  • Python Image Processing 1.0.6.v201712021523
  • Python Scripting Extension 3.1.0.v201602120920

plus the respective sources

I use nano so no issues with RTF.
Like I already mentioned, I can start the environment with the same command in the sh script by typing it in terminal. Therefore, py36_knime is definitely the actual name of the environment. Here is a paste from the .sh

#! /bin/bash
# start by making sure that the anaconda directory is on the PATH
# so that the source activate command works.
# This isn’t necessary if you already know that
# the anaconda bin dir is on the PATH
export PATH=“anaconda3/bin:$PATH”

source activate py36_knime
python “$@” 1>&1 2>&2

the name of the environment is py36_knime

running: source activate py36_knime on the command line initiates the environment.

Running sh knime_py36.sh also activates the environment.

Dear @Andrew_Bolton and @sunworshipper,

finally, we were able to reproduce the issue: the (rather misleading) error arises if the exported path does not point to the conda binary, i.e. the path you specify should be absolute. @sunworshipper, in your paste from above you point to “anaconda3/bin”, which points to a relative location (Relative to where the script lies? Where it is executed? I don’t really know actually). The execution still works from the CLI, because, unlike KNIME, your CLI knows where to find the command conda

However, if you put the absolute path (check e.g. via dirname $(which conda)), it should work also from within knime.

We hope that fixes your issues. We’ll also update our settings page and see if we can build a more descriptive error message. Let us know if it still persists. Thanks a ton for your feedback!
Best,
Lukas