Pip does not work with build_python_extension.py

[pirotex]

I faced a problem that some KNIME extensions raised a Maven error. After some investigations, I found that the problem is related to the conda environment yml files. Namely, If there is any pip package in a yml file, it fails every time.

The env files are attached. There are indeed derived from the mock extension, provided by KNIME team, the only difference is that
my_conda_env_ok.yml.txt (206 Bytes)
my_conda_env_failed.yml.txt (227 Bytes)

failed has an additional pip section with numpy.
The link to the extensions tutorial_extension_ok_and_failed.zip - Google Drive

How one can solve it? I need to include some packages that do not have a conda package, only pip.

[steffen_KNIME]

Hi @pirotex,

that could be due to an older version of your bundling environment, if named as in the tutorial, that is knime-ext-bundling.
Could you…
conda activate knime-ext-bundling
conda list knime
and post the version of knime-extension-bundling?

# Name                    Version                   Build  Channel
knime-extension-bundling  4.7.0                         4    knime

If it differs from the posted example here, then please update it to the latest (confusing, yes: the environment for bundling must be the latest one to support pip dependencies and thus is of version 4.7)
Update it for example via
conda update knime-extension-bundling -c knime -c conda-forge

Does that help?

Best regards
Steffen

[pirotex]

Hi Steffen,

Thanks for your advise! Unfortunately, upgrading to 4.7 does not help. The issue is the same

# packages in environment at /home/sergeyadmin/miniconda3/envs/knime-ext-bundling:                   │  File "/home/sergeyadmin/miniconda3/envs/knime-ext-bundling/bin/build_python_extension.py", line 363
#                                                                                                    │, in _run_maven_build
# Name                    Version                   Build  Channel                                   │    raise RuntimeError("Maven build failed.")
knime-extension-bundling  4.7.0                         4    knime

[steffen_KNIME]

Could you paste the whole error log above the [ERROR] in the right part of your screenshot please?
Thanks

[pirotex]

log_failed.txt (524.6 KB)
The full log is attached

[steffen_KNIME]

Hey,

I can reproduce the issue and will come back to you once we found a solution. This seems to be a MacOS issue.

Steffen

[pirotex]

Hi Steffen,

Thank you very much! But how can it be a MacOS issue if I tried to run it only on a Linux machine?

[steffen_KNIME]

Ah. I did a wrong assumption, I see.
Thanks for the correction.

[steffen_KNIME]

Hi @pirotex,

there could be two issues.

One is a library issue, which should be resolved in the nightly version of knime-extension-bundling, please try this nightly package for your bundling environment:

conda create -n knime-ext-bundling -c "knime/label/nightly" -c conda-forge knime-extension-bundling

The other is that in your example numpy is already included in the conda dependencies (i.e. in knime-python-base). If you specify the pip version of numpy for example to numpy==1.10, it should be put down as an additional pip dependency. As this is in no way easy to see, I filed a ticket for that (AP-20308).

Could you these two things out and tell me whether that worked?
Thanks!

Best regards
Steffen