Knime Python bundle not working

AD1529 · October 25, 2024, 5:10pm

Hi all,

I’m facing a problem I’m not able to solve.
I’m trying to create the bundle as I usually do.
But with knime 5.3 I have this error:

Command: [‘/opt/miniconda3/envs/knime-ext-bundling/bin/micromamba’, ‘create’, ‘–prefix’, ‘/var/folders/_h/vftmyhhj0j1b3596kb9bclmc0000gn/T/tmpxht0_n2v/prefix’, ‘–dry-run’, ‘–json’, ‘–override-channels’, ‘–channel’, ‘knime’, ‘–channel’, ‘conda-forge’, ‘–channel’, ‘file:///var/folders/_h/vftmyhhj0j1b3596kb9bclmc0000gn/T/tmpj507wie6’, ‘python 3.11.', ‘knime-extension >=5.3’, ‘knime-python-base >=5.3’, 'lightgbm 4.3.0.’, ‘markdown >=3.6.1’, ‘numpy 1.26.2.', 'pandas 2.0.3.’, ‘scikit-learn 1.4.2.', 'scipy 1.13.0.’, ‘shap 0.46.0.', 'typing-extensions 4.11.0.’]
STDOUT:
{
“solver_problems”: [
“unsupported request”,
“unsupported request”,
“package knime-python-base-5.3.0-py38_202406171400 requires python >=3.8.19,<3.8.20.0a0, but none of the providers can be installed”
],
“success”: false
}

I tried many times, but I always have this problem.
However, it’s weird since when creating the environment, python 11 is required:

conda create -n my_python_env python=3.11 knime-python-base=5.3 knime-
extension=5.3 -c knime -c conda-forge

could you please help me?

thanks
Daniela

MartinDDDD · October 26, 2024, 7:58am

I just copied your conda command and managed to successfully create one w/o any error (on windows).

Looking at your log:

am I right in assuming you are running on linux?
it is weird to see knime-ext-bundling in the error msg - as far as I know this is needed for bundling Python-based extensions and it’s not on the list of packages that are being installed when I create a new env => are you installing any additional packages?
i.e. are you just running:
conda create -n my_python_env python=3.11 knime-python-base=5.3 knime- extension=5.3 -c knime -c conda-forge
or something like:
conda create -n my_python_env python=3.11 knime-python-base=5.3 knime-extension=5.3 -c knime -c conda-forge knime-ext-bundling

AD1529 · October 26, 2024, 8:41am

I Martin,
thank you for your kind reply.

I’m running on macOs (M3).
Yes exactly, I’m bundling Python-based extensions.
I did it many times with Knime 5.2 but the new version is driving me a bit crazy. It is very slow in execution and I can no longer bundle the nodes I create.
These are the packages of the environment:

channels:

knime
conda-forge
dependencies:
python=3.11
knime-extension>=5.3
knime-python-base>=5.3
lightgbm=4.3.0
markdown>=3.6.1
numpy=1.26.2
pandas=2.0.3
scikit-learn=1.4.2
scipy=1.13.0
shap=0.46.0
typing-extensions=4.11.0

The only difference wrt to the bundling on version 5.2 is the Shap package I used to create an explanability node for ranking algorithms (the loop shap node already available on Knime is for classification)

and this is my history:

69 conda create -n findhr_nodes_env python=3.11 knime-python-base=5.3 knime-extension=5.3 -c knime -c conda-forge
70 conda activate findhr_nodes_env
71 conda create -n knime-ext-bundling -c knime -c conda-forge knime-extension-bundling=5.3
72 conda create -n knime-ext-bundling -c knime -c conda-forge knime-extension-bundling=5.3
73 conda activate knime-ext-bundling
74 build_python_extension.py /Users/dr/Desktop/Findhr_Knime/Knime_python_nodes/knime_nodes_extension /Users/dr/Desktop/Findhr_Bundle

71 and 72 are the same since I had an internet connection issue and I had to repeat the command.

And then I got the error I copied in the first message.

Any suggestion?
Thank you for your help.
Daniela

mlauber71 · October 26, 2024, 9:16am

@AD1529 I am not sure about the python 3.11 thing. In the past knime was using 3.9 you can extract the yaml file being used by knime like this

Might be worth a try. But also there are several threads in the forum complaining about the use of python so there might be something going on.

MartinDDDD · October 26, 2024, 9:23am

Hi Daniela,

thanks for the additional information.

I think I now understand that the error did not happen during installing your conda envs, but when you activated your knime-ext-bundling env and then want to bundle your extension.

Other than removing the knime-ext-bundling env using
conda env remove -n knime-ext-bundling
and creating it again to make sure your connection loss didn’t make something bug out the only other thing to test is if it is working when bundling w/o shap package… although in the logs you provided there does not seem to be anything indicating that this might be the cause…

AD1529 · October 29, 2024, 9:19pm

Hi Martin and mlauber71,

thank you for your replies.
I recreated all the nodes, everything works apart from the bundle.
This is the environment:

name: findhr_nodes_env

channels:

knime
conda-forge
dependencies:
python=3.11
knime-extension>=5.3
knime-python-base>=5.3
lightgbm=4.3.0
markdown>=3.6.1
numpy=1.26.2
pandas=2.0.3
scikit-learn=1.4.2
scipy=1.13.0
shap=0.46.0
typing-extensions=4.11.0

This is the error.
INFO:main:Filling template files from /opt/miniconda3/envs/knime-ext-bundling/bin/templates/python-extension-plugin
and storing the resulting plugin to
/var/folders/_h/vftmyhhj0j1b3596kb9bclmc0000gn/T/tmpop42s9qq/findhr.knime.nodes_extension/src/main/python.
Using bundle name ‘findhr.knime.nodes_extension.channel.bin’.
Using plugin template from ‘/opt/miniconda3/envs/knime-ext-bundling/bin/templates/conda-channel-plugin’.
Rendering plugin to ‘/var/folders/_h/vftmyhhj0j1b3596kb9bclmc0000gn/T/tmpop42s9qq/findhr.knime.nodes_extension.channel.bin’…
Using bundle name ‘findhr.knime.nodes_extension.channel.bin’.
Using fragment template from ‘/opt/miniconda3/envs/knime-ext-bundling/bin/templates/conda-channel-platform-fragment’.
Rendering fragment for linux-64 to ‘/var/folders/_h/vftmyhhj0j1b3596kb9bclmc0000gn/T/tmpop42s9qq/findhr.knime.nodes_extension.channel.bin.linux.x86_64’…
Running ‘conda-lock lock --micromamba --no-mamba --without-cuda -f /Users/danielarotelli/Desktop/Findhr_Knime/Knime_python_nodes/knime_nodes_extension/findhr_nodes_env.yml -c knime -c conda-forge -p linux-64 --lockfile /private/var/folders/_h/vftmyhhj0j1b3596kb9bclmc0000gn/T/tmprppvvjf1/lockfile’…
Running command failed with exit code 1. Standard output: 'Message key not found in json! returning the full json text
’ Standard error: '/opt/miniconda3/envs/knime-ext-bundling/lib/python3.9/site-packages/pydantic/_internal/_config.py:341: UserWarning: Valid config keys have changed in V2:

‘allow_mutation’ has been removed
warnings.warn(message, UserWarning)
Locking dependencies for [‘linux-64’]…
INFO:conda_lock.conda_solver:linux-64 using specs [‘python 3.11.', ‘knime-extension >=5.3’, ‘knime-python-base >=5.3’, 'lightgbm 4.3.0.’, ‘markdown >=3.6.1’, ‘numpy 1.26.2.', 'pandas 2.0.3.’, ‘scikit-learn 1.4.2.', 'scipy 1.13.0.’, ‘shap 0.46.0.', 'typing-extensions 4.11.0.’]
Could not lock the environment for platform linux-64
{‘solver_problems’: [‘unsupported request’, ‘unsupported request’, ‘package knime-python-base-5.3.0-py38_202406171400 requires python >=3.8.19,<3.8.20.0a0, but none of the providers can be installed’], ‘success’: False}
Command: [‘/opt/miniconda3/envs/knime-ext-bundling/bin/micromamba’, ‘create’, ‘–prefix’, ‘/var/folders/_h/vftmyhhj0j1b3596kb9bclmc0000gn/T/tmpkp8bibvr/prefix’, ‘–dry-run’, ‘–json’, ‘–override-channels’, ‘–channel’, ‘knime’, ‘–channel’, ‘conda-forge’, ‘–channel’, ‘file:///var/folders/_h/vftmyhhj0j1b3596kb9bclmc0000gn/T/tmpie2peqv3’, ‘python 3.11.', ‘knime-extension >=5.3’, ‘knime-python-base >=5.3’, 'lightgbm 4.3.0.’, ‘markdown >=3.6.1’, ‘numpy 1.26.2.', 'pandas 2.0.3.’, ‘scikit-learn 1.4.2.', 'scipy 1.13.0.’, ‘shap 0.46.0.', 'typing-extensions 4.11.0.’]
STDOUT:
{
“solver_problems”: [
“unsupported request”,
“unsupported request”,
“package knime-python-base-5.3.0-py38_202406171400 requires python >=3.8.19,<3.8.20.0a0, but none of the providers can be installed”
],
“success”: false
}

STDERR:
critical libmamba Could not solve for environment specs

Traceback (most recent call last):
File “/opt/miniconda3/envs/knime-ext-bundling/bin/conda-lock”, line 10, in
sys.exit(main())
File “/opt/miniconda3/envs/knime-ext-bundling/lib/python3.9/site-packages/click/core.py”, line 1157, in call
return self.main(args, **kwargs)
File “/opt/miniconda3/envs/knime-ext-bundling/lib/python3.9/site-packages/click/core.py”, line 1078, in main
rv = self.invoke(ctx)
File “/opt/miniconda3/envs/knime-ext-bundling/lib/python3.9/site-packages/click/core.py”, line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/opt/miniconda3/envs/knime-ext-bundling/lib/python3.9/site-packages/click/core.py”, line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File “/opt/miniconda3/envs/knime-ext-bundling/lib/python3.9/site-packages/click/core.py”, line 783, in invoke
return __callback(args, **kwargs)
File “/opt/miniconda3/envs/knime-ext-bundling/lib/python3.9/site-packages/click/decorators.py”, line 33, in new_func
return f(get_current_context(), args, **kwargs)
File “/opt/miniconda3/envs/knime-ext-bundling/lib/python3.9/site-packages/conda_lock/conda_lock.py”, line 1332, in lock
lock_func(
File “/opt/miniconda3/envs/knime-ext-bundling/lib/python3.9/site-packages/conda_lock/conda_lock.py”, line 1040, in run_lock
make_lock_files(
File “/opt/miniconda3/envs/knime-ext-bundling/lib/python3.9/site-packages/conda_lock/conda_lock.py”, line 381, in make_lock_files
lock_content = lock_content | create_lockfile_from_spec(
File “/opt/miniconda3/envs/knime-ext-bundling/lib/python3.9/site-packages/conda_lock/conda_lock.py”, line 787, in create_lockfile_from_spec
deps = _solve_for_arch(
File “/opt/miniconda3/envs/knime-ext-bundling/lib/python3.9/site-packages/conda_lock/conda_lock.py”, line 709, in _solve_for_arch
conda_deps = solve_conda(
File “/opt/miniconda3/envs/knime-ext-bundling/lib/python3.9/site-packages/conda_lock/conda_solver.py”, line 157, in solve_conda
dry_run_install = solve_specs_for_arch(
File “/opt/miniconda3/envs/knime-ext-bundling/lib/python3.9/site-packages/conda_lock/conda_solver.py”, line 369, in solve_specs_for_arch
proc.check_returncode()
File “/opt/miniconda3/envs/knime-ext-bundling/lib/python3.9/site-packages/conda_lock/_vendor/poetry/utils/_compat.py”, line 168, in check_returncode
raise CalledProcessError(
conda_lock._vendor.poetry.utils._compat.CalledProcessError: Command ‘[’/opt/miniconda3/envs/knime-ext-bundling/bin/micromamba’, ‘create’, ‘–prefix’, ‘/var/folders/_h/vftmyhhj0j1b3596kb9bclmc0000gn/T/tmpkp8bibvr/prefix’, ‘–dry-run’, ‘–json’, ‘–override-channels’, ‘–channel’, ‘knime’, ‘–channel’, ‘conda-forge’, ‘–channel’, ‘file:///var/folders/_h/vftmyhhj0j1b3596kb9bclmc0000gn/T/tmpie2peqv3’, 'python 3.11.', ‘knime-extension >=5.3’, ‘knime-python-base >=5.3’, 'lightgbm 4.3.0.', ‘markdown >=3.6.1’, 'numpy 1.26.2.‘, ‘pandas 2.0.3.', 'scikit-learn 1.4.2.’, ‘scipy 1.13.0.', 'shap 0.46.0.’, ‘typing-extensions 4.11.0.*’]’ returned non-zero exit status 1.

If I remove the version from dependencies

knime-extension
knime-python-base

The error becomes:

INFO:main:Filling template files from /opt/miniconda3/envs/knime-ext-bundling/bin/templates/python-extension-plugin
and storing the resulting plugin to
/var/folders/_h/vftmyhhj0j1b3596kb9bclmc0000gn/T/tmp8av9r_my/findhr.knime.nodes_extension/src/main/python.
Using bundle name ‘findhr.knime.nodes_extension.channel.bin’.
Using plugin template from ‘/opt/miniconda3/envs/knime-ext-bundling/bin/templates/conda-channel-plugin’.
Rendering plugin to ‘/var/folders/_h/vftmyhhj0j1b3596kb9bclmc0000gn/T/tmp8av9r_my/findhr.knime.nodes_extension.channel.bin’…
Using bundle name ‘findhr.knime.nodes_extension.channel.bin’.
Using fragment template from ‘/opt/miniconda3/envs/knime-ext-bundling/bin/templates/conda-channel-platform-fragment’.
Rendering fragment for linux-64 to ‘/var/folders/_h/vftmyhhj0j1b3596kb9bclmc0000gn/T/tmp8av9r_my/findhr.knime.nodes_extension.channel.bin.linux.x86_64’…
Running ‘conda-lock lock --micromamba --no-mamba --without-cuda -f /Users/danielarotelli/Desktop/Findhr_Knime/Knime_python_nodes/knime_nodes_extension/findhr_nodes_env.yml -c knime -c conda-forge -p linux-64 --lockfile /private/var/folders/_h/vftmyhhj0j1b3596kb9bclmc0000gn/T/tmptq2jyknk/lockfile’…
Running command failed with exit code 1. Standard output: 'Message key not found in json! returning the full json text
’ Standard error: '/opt/miniconda3/envs/knime-ext-bundling/lib/python3.9/site-packages/pydantic/_internal/_config.py:341: UserWarning: Valid config keys have changed in V2:

‘allow_mutation’ has been removed
warnings.warn(message, UserWarning)
Locking dependencies for [‘linux-64’]…
INFO:conda_lock.conda_solver:linux-64 using specs [‘python 3.11.', ‘knime-extension’, ‘knime-python-base’, 'lightgbm 4.3.0.’, ‘markdown >=3.6.1’, ‘numpy 1.26.2.', 'pandas 2.0.3.’, ‘scikit-learn 1.4.2.', 'scipy 1.13.0.’, ‘shap 0.46.0.', 'typing-extensions 4.11.0.’]
Could not lock the environment for platform linux-64
{‘solver_problems’: [‘unsupported request’, ‘unsupported request’], ‘success’: False}
Command: [‘/opt/miniconda3/envs/knime-ext-bundling/bin/micromamba’, ‘create’, ‘–prefix’, ‘/var/folders/_h/vftmyhhj0j1b3596kb9bclmc0000gn/T/tmp7_i75fwa/prefix’, ‘–dry-run’, ‘–json’, ‘–override-channels’, ‘–channel’, ‘knime’, ‘–channel’, ‘conda-forge’, ‘–channel’, ‘file:///var/folders/_h/vftmyhhj0j1b3596kb9bclmc0000gn/T/tmpix5p91f4’, ‘python 3.11.', ‘knime-extension’, ‘knime-python-base’, 'lightgbm 4.3.0.’, ‘markdown >=3.6.1’, ‘numpy 1.26.2.', 'pandas 2.0.3.’, ‘scikit-learn 1.4.2.', 'scipy 1.13.0.’, ‘shap 0.46.0.', 'typing-extensions 4.11.0.’]
STDOUT:
{
“solver_problems”: [
“unsupported request”,
“unsupported request”
],
“success”: false
}

STDERR:
critical libmamba Could not solve for environment specs

Traceback (most recent call last):
File “/opt/miniconda3/envs/knime-ext-bundling/bin/conda-lock”, line 10, in
sys.exit(main())
File “/opt/miniconda3/envs/knime-ext-bundling/lib/python3.9/site-packages/click/core.py”, line 1157, in call
return self.main(args, **kwargs)
File “/opt/miniconda3/envs/knime-ext-bundling/lib/python3.9/site-packages/click/core.py”, line 1078, in main
rv = self.invoke(ctx)
File “/opt/miniconda3/envs/knime-ext-bundling/lib/python3.9/site-packages/click/core.py”, line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/opt/miniconda3/envs/knime-ext-bundling/lib/python3.9/site-packages/click/core.py”, line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File “/opt/miniconda3/envs/knime-ext-bundling/lib/python3.9/site-packages/click/core.py”, line 783, in invoke
return __callback(args, **kwargs)
File “/opt/miniconda3/envs/knime-ext-bundling/lib/python3.9/site-packages/click/decorators.py”, line 33, in new_func
return f(get_current_context(), args, **kwargs)
File “/opt/miniconda3/envs/knime-ext-bundling/lib/python3.9/site-packages/conda_lock/conda_lock.py”, line 1332, in lock
lock_func(
File “/opt/miniconda3/envs/knime-ext-bundling/lib/python3.9/site-packages/conda_lock/conda_lock.py”, line 1040, in run_lock
make_lock_files(
File “/opt/miniconda3/envs/knime-ext-bundling/lib/python3.9/site-packages/conda_lock/conda_lock.py”, line 381, in make_lock_files
lock_content = lock_content | create_lockfile_from_spec(
File “/opt/miniconda3/envs/knime-ext-bundling/lib/python3.9/site-packages/conda_lock/conda_lock.py”, line 787, in create_lockfile_from_spec
deps = _solve_for_arch(
File “/opt/miniconda3/envs/knime-ext-bundling/lib/python3.9/site-packages/conda_lock/conda_lock.py”, line 709, in _solve_for_arch
conda_deps = solve_conda(
File “/opt/miniconda3/envs/knime-ext-bundling/lib/python3.9/site-packages/conda_lock/conda_solver.py”, line 157, in solve_conda
dry_run_install = solve_specs_for_arch(
File “/opt/miniconda3/envs/knime-ext-bundling/lib/python3.9/site-packages/conda_lock/conda_solver.py”, line 369, in solve_specs_for_arch
proc.check_returncode()
File “/opt/miniconda3/envs/knime-ext-bundling/lib/python3.9/site-packages/conda_lock/_vendor/poetry/utils/_compat.py”, line 168, in check_returncode
raise CalledProcessError(
conda_lock._vendor.poetry.utils._compat.CalledProcessError: Command ‘[’/opt/miniconda3/envs/knime-ext-bundling/bin/micromamba’, ‘create’, ‘–prefix’, ‘/var/folders/_h/vftmyhhj0j1b3596kb9bclmc0000gn/T/tmp7_i75fwa/prefix’, ‘–dry-run’, ‘–json’, ‘–override-channels’, ‘–channel’, ‘knime’, ‘–channel’, ‘conda-forge’, ‘–channel’, ‘file:///var/folders/_h/vftmyhhj0j1b3596kb9bclmc0000gn/T/tmpix5p91f4’, 'python 3.11.', ‘knime-extension’, ‘knime-python-base’, 'lightgbm 4.3.0.', ‘markdown >=3.6.1’, 'numpy 1.26.2.‘, ‘pandas 2.0.3.', 'scikit-learn 1.4.2.’, ‘scipy 1.13.0.', 'shap 0.46.0.’, ‘typing-extensions 4.11.0.*’]’ returned non-zero exit status 1.
’

If the problem is really python version, why in the guide the command to create the environment is:
conda create -n my_python_env python=3.11 knime-python-base=5.3 knime-
extension=5.3 -c knime -c conda-forge

Thanks for your help,
Daniela

MartinDDDD · October 30, 2024, 12:41pm

Hi Daniela,

have to admit I am a bit out of my depth here. Given that you definitely have python 3.11 installed I think it will likely be down to package compatibility issues.

I notice that it’s referencing the linux-64 build that is failing. It might be that your packages and their versions work for WIndows and MacOS builds, but for some reason not for linux.

So I have no idea how to possibly debug that …

One question (again). If you build without that shap package - is it working? Does it build successfully?

mlauber71 · October 30, 2024, 1:37pm

@AD1529 my recommendation would be to start out with the ‘official’ YAML configuration of KNIME and then expand from there by adding packages like lightgbm. Currently KNIME seems to best work with Python 3.9

name: py39_knime         # Name of the created environment
channels:                # Repositories to search for packages
- conda-forge
- knime                  # conda search knime-python-base -c knime --info
dependencies:            # List of packages that should be installed
- python=3.9             # Python
- knime-python-base      # dependencies of KNIME - Python integration
- py4j                   # used for KNIME <-> Python communication
- nomkl                  # Prevents the use of Intel's MKL
- pandas                 # Table data structures
- jedi=0.18.1            # Python script autocompletion
- python-dateutil        # Date and Time utilities
- numpy                  # N-dimensional arrays
- cairo                  # SVG support
- pillow                 # Image inputs/outputs
- matplotlib             # Plotting
- pyarrow=6.0            # Arrow serialization
- pyarrow-hotfix         # PyArrow Hotfix
- IPython                # Notebook support
- nbformat               # Notebook support
- scipy                  # Notebook support
- python-flatbuffers<2.0 # because tensorflow expects a version before 2
- h5py<3.0 # must be < 3.0 because they changed whether str or byte is returned
- protobuf>3.12          # Lower protobuf versions do not work with TensorFlow 2
- libiconv               # MDF Reader node
- asammdf=5.19.14        # MDF Reader node
- JPype1 # Databases

Typically I would then remove the restriction on Pyarrow since this seems outdated.

Once you have a conda environment up and running that does work with the basic KNIME nodes you can add more packages to the fray and remove restrictions.

conda env update -n py39_knime --file=“/Users/knime/py39_knime.yml”

Then one remark: if you have large blocks of code or Logs. Please attach them in a file or use the three backticks as code markers to make this more readabled.

steffen_KNIME · October 30, 2024, 7:43pm

Dear @AD1529,

in your log (please format as @mlauber71 mentioned, using ```), we have:

package knime-python-base-5.3.0-py38_202406171400 requires python >=3.8.19,<3.8.20.0a0, but none of the providers can be installed

You are saying you used the following command:

conda create -n my_python_env python=3.11 knime-python-base=5.3 knime-
extension=5.3 -c knime -c conda-forge

I double-checked, there is also a knime-python-base package of version 5.3 available compatible for python 3.11 on linux: https://anaconda.org/knime/knime-python-base/5.3.0/download/linux-64/knime-python-base-5.3.0-py311_202406171409.tar.bz2 (in this repository: Files | Anaconda.org )

It seems to me from the part copied from you log, that you do not have the correct knime-python-base packages available. Could it be that you are behind a proxy? If yes, maybe downloading the packages you need (knime-python-base 5.3 for all OS and py311) and an adjustment of your command to

conda create -n my_python_env python=3.11 knime-python-base=5.3 knime-
extension=5.3 -c knime -c conda-forge -c /somehow/the/path/to/the/downloaded/packages

could help moving forward.

Does anybody of this help you already?

Best
Steffen

ricciV1 · November 1, 2024, 9:50am

Hey @steffen_KNIME @mlauber71 @knime ,
I am running into the same issue mentioned above. (I am working on win-64)
Excerpt:

Could not lock the environment for platform linux-64
...
 STDOUT:
{
    "solver_problems": [
        "unsupported request",
        "unsupported request"
    ],
    "success": false
}

    STDERR:
critical libmamba Could not solve for environment specs

As part of troubleshooting I looked into the bundling environment created with conda create -n knime-ext-bundling -c knime -c conda-forge knime-extension-bundling=5.3
I saw that the python version in this env is: 3.9
Upon trying to upgrade it to 3.11.9 I get the following error message:

This seems to be the issue for me (… i guess?)

I tried to create a new bundling environment with conda create -n knime-ext-bundling -c knime -c conda-forge knime-extension-bundling=5.3 python=3.11 … but still the same message

My environment.yml looks like this and was created with conda env export --from-history | findstr -v “prefix” > environment.yaml

name: knime_python_env
channels:
  - defaults
dependencies:
  - knime-extension==5.3.0
  - knime-python-base==5.3.0
  - python==3.11.9
  - selenium==4.25.0
  - webdriver-manager
  - pillow
  - beautifulsoup4

steffen_KNIME · November 1, 2024, 2:46pm

Hi @ricciV1 and @AD1529,

I should have examined your statement closer:

@AD1529 said

If the problem is really python version, why in the guide the command to create the environment is:
conda create -n my_python_env python=3.11 knime-python-base=5.3 knime-
extension=5.3 -c knime -c conda-forge

Well… the guide never said that.

It says that you have to create an extra environment for bundling:

conda create -n knime-ext-bundling -c knime -c conda-forge knime-extension-bundling=5.3

There is nothing about knime-python-base=5.3 written there. Use the above command as it is written in the guide and tell me whether you still encounter issues.

Best
Steffen

PS: @ricciV1 thanks for sharing your OS - the bundling in general does happen for all (i.e. win, linux, mac, mac aarm) OS. And it always starts with linux, that’s why the error message mentions linux.

EDIT: PPS: thanks for providing all these details @ricciV1. Indeed knime-extension-bundling requires Python 3.9. We see no need to change that.

AD1529 · November 1, 2024, 3:31pm

Hi Steffen, thank you for your reply.
However, please have a look at the guide: Create a New Python based KNIME Extension

I followed all the steps as I usually did in version 5.2
I repeated the same steps from the beginning to the end. And I always have the same errors.

AD1529 · November 1, 2024, 3:39pm

Ps. I had done the same thing as @ricciV1 in order to solve the problem and I encountered the same errors again.

However, I do not understand why everything worked in 5.2 and now nothing works in 5.3. I have the same nodes (apart from one - that I also removed as suggested by @MartinDDDD) and I started from the official YAML as suggested by @mlauber71. I followed all the steps as indicated in the guide from the beginning to the bundling. I see all my 9 nodes in the node’s list, but if I try to bundle them, I get that error.

ricciV1 · November 1, 2024, 5:12pm

Hi @steffen_KNIME @AD1529
I have tried to set up the bundling env with your command: conda create -n knime-ext-bundling -c knime -c conda-forge knime-extension-bundling=5.3

But as expected I ran into the same issue.

Other question: why do you see no need to upgrade to 3.11?
It seems to be the issue (at least for me…i think).
Downgrading to 3.9 from 3.11 would not be really be an option

Kind regards Ricci

AD1529 · November 1, 2024, 6:31pm

Thank you Martin, but unfortunately not.

AD1529 · November 1, 2024, 6:39pm

Thank you Markus. I also downgraded to Python 3.9 but it does not work. Moreover in the guide (both for V5.2 and V5.3) there was Python 3.11 in the environment.

Since we are two people to have the same issue, with Win and Mac, I really do not know where the solution could be.

steffen_KNIME · November 2, 2024, 12:44am

Things are getting mixed up here. The screenshot you, @AD1529, shared, shows a different command than the one you claim it would.

Your command from the original post uses the package knime-extension-bundling, whereas the screenshot you are referring to uses the package knime-extension.

Please keep in mind that you are generating two different environments for two entirely different purposes.

The environment which is described in the screenshot is for the development process of your extension. The command copied from step 4 of the tutorial (your screenshot) is conda create -n my_python_env python=3.11 knime-python-base=5.3 knime-extension=5.3 -c knime -c conda-forge
Again, please note knime-extension, not knime-extension-bundling. This command can have different Python versions because the environment you want to use can have different Python versions. This is up to you, because you know best what Python version your environment needs to have.
The environment I have been talking about, because you always referred to knime-extension-bundling in your commands, is for the bundling of your extension. It is not at all connected to the environment you use for your extension. It is used to bundle your extension so that you can share your extension with others. The command copied from Sharing your Extension - Option 1, step 1 is conda create -n knime-ext-bundling -c knime -c conda-forge knime-extension-bundling=5.3
This is the command I talked about. This command has Python 3.9 set, because we tested it with this version and this environment does not have to be altered at all. There is just no need to have other Python versions.

@AD1529 the title of this conversation is “…bundle not working”. I assumed you are trying to bundle your extension, for which you would use the command of the second environment. Is that not correct?

Lastly, I just tested the command (for completeness again: conda create -n knime-ext-bundling -c knime -c conda-forge knime-extension-bundling=5.3) on Mac, and it worked.
@AD1529 @ricciV1 is it this very exact command which does not work for you (Yes you said so, @ricciV1, but there seems to be enough confusion that I want to be sure again)?

Steffen

ricciV1 · November 2, 2024, 9:54am

Hey @steffen_KNIME ,

yes I think there is much confusion.
I failed to mention that my issue did NOT occur whilst setting up the bundling environment but whilst executing the following command:

build_python_extension.bat <path/to/directoryof/myextension/> <path/to/directoryof/output>

This command is the next after setting up the environment. Mentioned here.

To clear all confusion from my side:

setting up the bundling environment works fine!
executing the command for bundling (see above) does not work fine and yields in the issue mentioned in my other comment (above)

Could not lock the environment for platform linux-64
...
 STDOUT:
{
    "solver_problems": [
        "unsupported request",
        "unsupported request"
    ],
    "success": false
}

    STDERR:
critical libmamba Could not solve for environment specs

Refering to my original comment: Knime Python bundle not working - #10 by ricciV1

steffen_KNIME · November 2, 2024, 4:08pm

Hi @ricciV1,

ah, thanks for explaining. My thoughts for your scenario:

Your env.yml does not contain the channels conda-forge and knime, (see the example in the guide) - could you add them and try again?
The log you posted in your original post, was there anything more to it? If yes, could you share the whole log in a .txt file please?
And I hope it became clear that the Python version is irrelevant in your case

Thanks
Steffen

AD1529 · November 2, 2024, 5:15pm

Hi @steffen_KNIME,

find attached all the screenshots of my project.

This is the structure:

These are the created nodes:

This is the knime.yml file

This is the config.yml file:

This is the environment:

This is the error:
error_knime.txt (6.4 KB)

I also attach the history, so you can check that I followed the instructions:

Can I do something else to make you understand where the problem is?

Thank you for your help.
Daniela