Need to be able to use many Anaconda environments with KNIME

The following article shows how to set up an Anaconda environment::
https://www.knime.com/blog/setting-up-the-knime-python-extension-revisited-for-python-30-and-20
The problem is that it restricts us to one Anaconda environment for Python 3. I need the ability to specify an Anaconda environment for each Python 3 node. Same idea for R nodes. Need this for both KNIME AP and for KNIME Server. When will the feature be available?

1 Like

Hi CarlosBarichello,

We already received that feature request recently. It’s good to know that you’re also interested in it. This is in our pipeline and will be under development soon. I can’t give you any specific timeline, unfortunately.

Marcel

1 Like

Hi Marcel,

Glad to hear that this will be under development soon. This feature will enable us to expand our use of KNIME Server at our company. Currently we are limited to running one Python 3 application on our KNIME server.

We are not able to have a generic Python 3 environment on our KNIME server for all applications because each application has very specific needs for their environment. For example, one of our applications has to use a different version of Pandas than the other application, so they cannot share a Conda environment. Another example is that one of our applications requires Python 3.7 while the other requires Python 3.6.

What would be really useful if each Python (or R) node allowed us to type in a Conda environment name instead of making us select “Python 2” or “Python 3”. This approach would also remove the need for us to have to create a shell script (as described in the article I mentioned) to source the Conda environment.

Thanks,

Carlos -

3 Likes

Hello @CarlosBarichello,

Understood and totally makes sense. We try to add the possibility to control the used conda environment per node as soon as possible.

Christian

1 Like

Hi CarlosBarichello,

With the 3.7 release of KNIME Analytics Platform, we added some first functionality to control Python environments per node via flow variables.

I’ve attached a simple example workflow that illustrates the new functionality: python-command-via-flow-variable.knwf (18.7 KB).

This approach still requires to specify paths to shell scripts. We aim to simplify the configuration of conda environments in the future (e.g., as you suggested, by allowing to simply specify conda environment names via the node dialog).

Marcel

3 Likes

Hi Marcel,

I download KNIME AP 3.7 and tested using python-command-via-flow-variable.knwf. This allowed us to specify a different conda environment for each python node and it solves our problem for Python. Thanks.

I assume that the latest version of KNIME Server will support this feature as well?

Also looking forward to seeing this work with R Nodes.

Thanks,

Carlos -

Hi,

Is there any development on this? Is this feature part of the newest KNIME Server?

Has this feature been deployed to KNIME Server? If so, what version of KNIME Server?

Hi mamor and CarlosBarichello,

Sorry for the delay on this. The mentioned feature was introduced with KNIME Analytics Platform 3.7 and should therefore work for all KNIME Server installations that are compatible with that AP version.

Marcel

3 Likes