Conda and Python configuration for the KNIME Server

KNIME Server
#1

Hi everyone,

We’re trying to build a KNIME Server 4.9.1 and Distributed executors 4.0.1 array with rabbitmq instance.
I’ve got the issue setting up the python integration.
So I’ve set anaconda to /apps/anaconda3 and created two environments in it: py27_knime and py35_knime.
I’ve set the following settings to the executor’s configuration/config.ini file:

org.knime.python2/condaDirectoryPath=/apps/anaconda3
org.knime.python2/defaultPythonOption=python3
org.knime.python2/python2CondaEnvironmentName=py27_knime
org.knime.python2/python2Path=python
org.knime.python2/python3CondaEnvironmentName=py35_knime
org.knime.python2/python3Path=python3
org.knime.python2/pythonEnvironmentType=conda
org.knime.python2/serializerId=org.knime.serialization.flatbuffers.column

The following error messages appears in the executor’s log:

2020-02-06 14:10:20,351 : ERROR : KNIME-Worker-0 : 9cede91d-97a8-4949-9424-5f6437b587de : Node : Python Script (1⇒1) : 0:4 : Execute failed: Could not start Python kernel. Error during Python installation test: Library numpy is missing.
Library pandas is missing, required minimum version is 0.20.0 (inclusive).
/apps/knime/knime_executor_current/plugins/org.knime.python2.envconfigs_4.0.0.v201906042152/envconfigs/linux/start_py.sh: line 13: /home/kserveraz/anaconda3/etc/profile.d/conda.sh: No such file or directory
/apps/knime/knime_executor_current/plugins/org.knime.python2.envconfigs_4.0.0.v201906042152/envconfigs/linux/start_py.sh: line 14: conda: command not found

kserveraz is my OS username for installation, since I installed anaconda into the custom location /apps/anaconda3, it is not supposed to have anything in /home/kserveraz/anaconda3. It looks like the python plugin ignores my python and conda setup.
Can someone give me a clue with this issue, please?

Thanks
Andriy

#2

Hi,
is anaconda installed on the distributed executors and the environments created there as well? Also: in the KNIME Server Administration Guide the file configuration/config.ini is never mentioned. Are you sure you set this up correctly?
Kind regards
Alexander

2 Likes
#3

Hi Alexander,

Yes, the anaconda and environments have been installed on the executor host.
For the second question, no server config file has been mentioned to contain python settings in the docs, so I’m really not sure where to set them. https://docs.knime.com/2019-12/server_admin_guide/index.html#server-side-setup is a bit different topic.

Thanks
Andriy

#4

Hi,
you have to export an Eclipse preference file with the correct settings and then put that in the appropriate folder on the server. On the executor you can then use those settings as described in this section: https://docs.knime.com/2019-12/server_admin_guide/index.html#client-side-setup
Kind regards
Alexander

4 Likes
#5

Hi Alexander,

Thanks to your hints, I was able to setup the client profiles from the server side, and now I can have a server as a central point of executor array configuration. Also I have discovered how these settings are stored on the executor’s side, so I can do that as well if needed.

Thanks a lot again.
Andriy

4 Likes
#6

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.