UnicodeDecodeError When Bundling Pure-Python Extension

DerMaxdorfer · August 16, 2023, 12:58pm

Hi KNIMErs,

I’m trying to bundle a Python-based extension for KAP v5.1.0 from a Windows machine, but keep running into a UnicodeDecodeError:

(knime_5.1.0_extension_bundling) C:\>build_python_extension.bat C:\Users\[MY USERNAME]\Documents\KNIME\Extensions\[my extension folder] C:\Users\[MY USERNAME]\Desktop\[my extension name]_v1.0.0_knime5.1.0

(knime_5.1.0_extension_bundling) C:\>python "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\build_python_extension.py" C:\Users\[MY USERNAME]\Documents\KNIME\Extensions\[my extension folder] C:\Users\[MY USERNAME]\Desktop\[my extension name]_v1.0.0_knime5.1.0
INFO:__main__:Filling template files from C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\templates\python-extension-plugin
and storing the resulting plugin to
C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpycz0kupf\[my extension name]\src\main\python.
Using bundle name '[my extension name].channel.bin'.
Using plugin template from 'C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\templates\conda-channel-plugin'.
Rendering plugin to 'C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpycz0kupf\[my extension name].channel.bin'...
Using bundle name '[my extension name].channel.bin'.
Using fragment template from 'C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\templates\conda-channel-platform-fragment'.
Rendering fragment for linux-64 to 'C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpycz0kupf\[my extension name].channel.bin.linux.x86_64'...
Running 'conda-lock lock --micromamba --no-mamba -f C:\Users\[MY USERNAME]\Documents\KNIME\Extensions\[my extension folder]\conda_env.yml -c conda-forge -c knime -p linux-64 --lockfile C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpxkgsbinx\lockfile'...
Traceback (most recent call last):
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\build_python_extension.py", line 524, in <module>
    build_extension(
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\build_python_extension.py", line 72, in build_extension
    _build_extension(
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\build_python_extension.py", line 119, in _build_extension
    bundles += _create_conda_channels(render_dir_path, force, config)
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\build_python_extension.py", line 352, in _create_conda_channels
    return bundler.bundle()
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\bundle_channels.py", line 81, in bundle
    self._bundle_fragment(
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\bundle_channels.py", line 114, in _bundle_fragment
    gather_env_specs(
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\gather_env_specs.py", line 132, in gather_env_specs
    _perform_with_conda_lock(
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\gather_env_specs.py", line 41, in _perform_with_conda_lock
    lockfile_dict = _run_conda_lock(env_yaml, platform, channels, lockfile)
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\gather_env_specs.py", line 79, in _run_conda_lock
    run_and_capture(command)
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\utils.py", line 128, in run_and_capture
    + f" Standard error: '{output.stderr.decode()}'"
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfb in position 1864: invalid start byte

Based on the exception above, I believe that the problem is somehow related to my conda_env.yml. It is UTF-8 encoded and follows the example of the official guide:

name: [my extension's environment name]
channels:
  - conda-forge
  - knime
dependencies:
  - python=3.11
  - knime-extension=5.1
  - knime-python-base=5.1
  - pip:
      - [a pip package]

I have been racking my brains over this for hours now, so any help in fixing this will be much appreciated.

Cheers!

nemad · August 17, 2023, 3:36pm

Hello @DerMaxdorfer ,

the actual error seems to be obscured by an encoding issue.
The script assumes that the process output encoding is UTF-8 but it seems to be something else.
This is a bug on our end because we assume UTF-8 but I think you might be able to work around this problem by setting the environment variable PYTHONIOENCODING to utf8 before running the bundling:

export PYTHONIOENCODING=utf8
build_python_extension.bat ...

I hope this helps to get to the actual error cause.
Best regards,

Adrian

DerMaxdorfer · August 18, 2023, 9:07am

Hi @nemad ,

Thanks for looking into this!

You were absolutely right that setting the env variable would solve the encoding issue. However, it still seems odd to me because all my extension’s files are already UTF-8 encoded.

As you expected, fixing this revealed a different error:

(knime_5.1.0_extension_bundling) C:\>conda env config vars list

(knime_5.1.0_extension_bundling) C:\>conda env config vars set PYTHONIOENCODING=utf8
To make your changes take effect please reactivate your environment

(knime_5.1.0_extension_bundling) C:\>conda activate knime_5.1.0_extension_bundling

(knime_5.1.0_extension_bundling) C:\>conda env config vars list
PYTHONIOENCODING = utf8

(knime_5.1.0_extension_bundling) C:\>build_python_extension.bat C:\Users\[MY USERNAME]\Documents\KNIME\Extensions\[my extension folder] C:\Users\[MY USERNAME]\Desktop\[my extension name]_v1.0.0_knime5.1.0

(knime_5.1.0_extension_bundling) C:\>python "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\build_python_extension.py" C:\Users\[MY USERNAME]\Documents\KNIME\Extensions\[my extension folder] C:\Users\[MY USERNAME]\Desktop\[my extension name]_v1.0.0_knime5.1.0
INFO:__main__:Filling template files from C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\templates\python-extension-plugin
and storing the resulting plugin to
C:\Users\[MY USERNAME]\AppData\Local\Temp\tmp7sdi9sy4\[my extension name]\src\main\python.
Using bundle name '[my extension name].channel.bin'.
Using plugin template from 'C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\templates\conda-channel-plugin'.
Rendering plugin to 'C:\Users\[MY USERNAME]\AppData\Local\Temp\tmp7sdi9sy4\[my extension name].channel.bin'...
Using bundle name '[my extension name].channel.bin'.
Using fragment template from 'C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\templates\conda-channel-platform-fragment'.
Rendering fragment for linux-64 to 'C:\Users\[MY USERNAME]\AppData\Local\Temp\tmp7sdi9sy4\[my extension name].channel.bin.linux.x86_64'...
Running 'conda-lock lock --micromamba --no-mamba -f C:\Users\[MY USERNAME]\Documents\KNIME\Extensions\[my extension folder]\conda_env.yml -c conda-forge -c knime -p linux-64 --lockfile C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpkcn9a4pj\lockfile'...
Running command failed with exit code 1. Standard output: '' Standard error: 'C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\pydantic\_internal\_config.py:269: UserWarning: Valid config keys have changed in V2:
* 'allow_mutation' has been removed
  warnings.warn(message, UserWarning)
Traceback (most recent call last):
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\urllib3\connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\urllib3\connectionpool.py", line 386, in _make_request
    self._validate_conn(conn)
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\urllib3\connectionpool.py", line 1042, in _validate_conn
    conn.connect()
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\urllib3\connection.py", line 419, in connect
    self.sock = ssl_wrap_socket(
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\urllib3\util\ssl_.py", line 449, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\urllib3\util\ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\ssl.py", line 501, in wrap_socket
    return self.sslsocket_class._create(
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\ssl.py", line 1041, in _create
    self.do_handshake()
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\ssl.py", line 1310, in do_handshake
    self._sslobj.do_handshake()
ConnectionResetError: [WinError 10054] Une connexion existante a dû être fermée par l’hôte distant

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\requests\adapters.py", line 486, in send
    resp = conn.urlopen(
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\urllib3\connectionpool.py", line 787, in urlopen
    retries = retries.increment(
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\urllib3\util\retry.py", line 550, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\urllib3\packages\six.py", line 769, in reraise
    raise value.with_traceback(tb)
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\urllib3\connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\urllib3\connectionpool.py", line 386, in _make_request
    self._validate_conn(conn)
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\urllib3\connectionpool.py", line 1042, in _validate_conn
    conn.connect()
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\urllib3\connection.py", line 419, in connect
    self.sock = ssl_wrap_socket(
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\urllib3\util\ssl_.py", line 449, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\urllib3\util\ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\ssl.py", line 501, in wrap_socket
    return self.sslsocket_class._create(
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\ssl.py", line 1041, in _create
    self.do_handshake()
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\ssl.py", line 1310, in do_handshake
    self._sslobj.do_handshake()
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(10054, 'Une connexion existante a dû être fermée par l’hôte distant', None, 10054, None))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\Scripts\conda-lock-script.py", line 9, in <module>
    sys.exit(main())
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\click\core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\click\core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\click\core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\click\core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\click\core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\click\decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\conda_lock\conda_lock.py", line 1332, in lock
    lock_func(
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\conda_lock\conda_lock.py", line 1040, in run_lock
    make_lock_files(
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\conda_lock\conda_lock.py", line 328, in make_lock_files
    lock_spec = make_lock_spec(
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\conda_lock\src_parser\__init__.py", line 89, in make_lock_spec
    lock_specs = _parse_source_files(src_files, platforms)
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\conda_lock\src_parser\__init__.py", line 70, in _parse_source_files
    desired_envs.append(parse_environment_file(src_file, platforms))
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\conda_lock\src_parser\environment_yaml.py", line 127, in parse_environment_file
    dep_map = {
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\conda_lock\src_parser\environment_yaml.py", line 128, in <dictcomp>
    platform: _parse_environment_file_for_platform(content, category, platform)
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\conda_lock\src_parser\environment_yaml.py", line 79, in _parse_environment_file_for_platform
    dependencies.append(parse_python_requirement("pip", manager="conda"))
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\conda_lock\src_parser\pyproject_toml.py", line 421, in parse_python_requirement
    conda_dep_name = normalize_pypi_name(name)
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\conda_lock\src_parser\pyproject_toml.py", line 78, in normalize_pypi_name
    if cname in get_lookup():
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\conda_lock\lookup.py", line 53, in get_forward_lookup
    return LOOKUP_OBJECT.pypi_lookup
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\functools.py", line 993, in __get__
    val = self.func(instance)
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\conda_lock\lookup.py", line 33, in pypi_lookup
    res = requests.get(self._mapping_url)
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\requests\api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\requests\api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\requests\sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\requests\sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\requests\adapters.py", line 501, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(10054, 'Une connexion existante a dû être fermée par l’hôte distant', None, 10054, None))

Just in case: The French part of the ConnectionResetError translates to An existing connection was forcibly closed by the remote host.

Best

nemad · August 21, 2023, 9:57am

Great, now we can look into the actual error.
There are multiple possible causes for this kind of error but since you already tried multiple times and could reproduce it reliably, I would rule out network and server load issues.
Are you by chance in a company network with a special firewall and/or proxy setup?

Best,
Adrian

DerMaxdorfer · August 21, 2023, 11:57am

Yes, I am in a company network indeed.

Can you tell me with which website(s) exactly the script is trying to establish a connection? I am asking because I already dealt with a bunch of SSL errors before reaching the current state.

nemad · August 25, 2023, 1:29pm

The line numbers don’t fit in my environment but judging from the error log it might be the following:
https://raw.githubusercontent.com/regro/cf-graph-countyfair/master/mappings/pypi/grayskull_pypi_mapping.yaml
You can double-check by going to "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\lib\site-packages\conda_lock\lookup.py" and looking for the value of self._mapping_url in the _LookupLoader class.

DerMaxdorfer · August 25, 2023, 7:21pm

Thanks, @nemad. This is one of the URLs that caused an SSL error on my side before I reached the UnicodeDecodeError, for which I ended up opening this topic initially. So, it’s already accessible from within the company network now.

Have you been able to investigate any other possible root causes and solutions?

PS: I just tried to bundle a dummy extension on my private MacBook, and it worked like a charm. Unfortunately, I still need to get it to run on my company’s device…

nemad · August 28, 2023, 3:41pm

Hm, so you can click the link and it opens in your browser?
Unfortunately, that doesn’t necessarily mean that it will also work for the requests Python package that is used by the script.
You can control the proxy settings via environment variables HTTP_PROXY and HTTPS_PROXY, though.
Here is a StackOverflow thread with a few more details: Proxies with Python 'Requests' module - Stack Overflow

DerMaxdorfer · August 29, 2023, 12:26pm

Hi @nemad!

Yes, exactly. I can open the link in my browser.

Thanks for the hint with the proxy settings. I knew that it existed, but never had to use it before.

Now, I am sure that you are going to laugh about this part:
Before meddling with the proxy settings, I gave it one last shot with the default ones. You can imagine my surprise when the script suddenly started to work through the part where I had been stuck all this time. The only difference to my previous attempts is basically that I am working on my company’s premises today. It failed before while working from home both with and without an active VPN connection…

However, the extension has still not bundled itself:

(knime_5.1.0_extension_bundling) C:\>build_python_extension.bat C:\Users\[MY USERNAME]\Documents\KNIME\Extensions\[my extension folder] C:\Users\[MY USERNAME]\Desktop\[my extension name]_v1.0.0_knime5.1.0

(knime_5.1.0_extension_bundling) C:\>python "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\build_python_extension.py" C:\Users\[MY USERNAME]\Documents\KNIME\Extensions\[my extension folder] C:\Users\[MY USERNAME]\Desktop\[my extension name]_v1.0.0_knime5.1.0
INFO:__main__:Filling template files from C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\templates\python-extension-plugin
and storing the resulting plugin to
C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpamg4yyt_\[my extension name]\src\main\python.
Using bundle name '[my extension name].channel.bin'.
Using plugin template from 'C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\templates\conda-channel-plugin'.
Rendering plugin to 'C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpamg4yyt_\[my extension name].channel.bin'...
Using bundle name '[my extension name].channel.bin'.
Using fragment template from 'C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\templates\conda-channel-platform-fragment'.
Rendering fragment for linux-64 to 'C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpamg4yyt_\[my extension name].channel.bin.linux.x86_64'...
Running 'conda-lock lock --micromamba --no-mamba -f C:\Users\[MY USERNAME]\Documents\KNIME\Extensions\[my extension folder]\conda_env.yml -c conda-forge -c knime -p linux-64 --lockfile C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpkpgnr_o3\lockfile'...
Using bundle name '[my extension name].channel.bin'.
Using fragment template from 'C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\templates\conda-channel-platform-fragment'.
Rendering fragment for win-64 to 'C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpamg4yyt_\[my extension name].channel.bin.win32.x86_64'...
Running 'conda-lock lock --micromamba --no-mamba -f C:\Users\[MY USERNAME]\Documents\KNIME\Extensions\[my extension folder]\conda_env.yml -c conda-forge -c knime -p win-64 --lockfile C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpproduxwr\lockfile'...
Using bundle name '[my extension name].channel.bin'.
Using fragment template from 'C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\templates\conda-channel-platform-fragment'.
Rendering fragment for osx-64 to 'C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpamg4yyt_\[my extension name].channel.bin.macosx.x86_64'...
Running 'conda-lock lock --micromamba --no-mamba -f C:\Users\[MY USERNAME]\Documents\KNIME\Extensions\[my extension folder]\conda_env.yml -c conda-forge -c knime -p osx-64 --lockfile C:\Users\[MY USERNAME]\AppData\Local\Temp\tmp_34gio9t\lockfile'...
Using bundle name '[my extension name].channel.bin'.
Using fragment template from 'C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\templates\conda-channel-platform-fragment'.
Rendering fragment for osx-arm64 to 'C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpamg4yyt_\[my extension name].channel.bin.macosx.aarch64'...
Running 'conda-lock lock --micromamba --no-mamba -f C:\Users\[MY USERNAME]\Documents\KNIME\Extensions\[my extension folder]\conda_env.yml -c conda-forge -c knime -p osx-arm64 --lockfile C:\Users\[MY USERNAME]\AppData\Local\Temp\tmp8epoqo85\lockfile'...
Using plugin template from 'C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\templates\python-extension-feature'.
Rendering feature to 'C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpamg4yyt_\[my extension name features]'...
INFO:__main__:Filling template files from C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\templates\python-extension-update-site to C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpamg4yyt_\[my extension name update].
INFO:__main__:Running maven build 'mvn.cmd package'...
Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/tycho/tycho-build/2.7.4/tycho-build-2.7.4.pom
[WARNING] Failed to read extensions descriptor C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpamg4yyt_\.mvn\extensions.xml: Plugin org.eclipse.tycho:tycho-build:2.7.4 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.eclipse.tycho:tycho-build:jar:2.7.4
[INFO] Scanning for projects...
Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/tycho/tycho-maven-plugin/2.7.5/tycho-maven-plugin-2.7.5.pom
[ERROR] [ERROR] Some problems were encountered while processing the POMs:
[ERROR] Child module C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpamg4yyt_\[my extension name]\pom.xml of C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpamg4yyt_\pom.xml does not exist @
[ERROR] Child module C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpamg4yyt_\[my extension name].channel.bin\pom.xml of C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpamg4yyt_\pom.xml does not exist @
[ERROR] Child module C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpamg4yyt_\[my extension name features]\pom.xml of C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpamg4yyt_\pom.xml does not exist @
[ERROR] Child module C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpamg4yyt_\[my extension name update]\pom.xml of C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpamg4yyt_\pom.xml does not exist @
[ERROR] Unresolveable build extension: Plugin org.eclipse.tycho:tycho-maven-plugin:2.7.5 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.eclipse.tycho:tycho-maven-plugin:jar:2.7.5 @
 @
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]
[ERROR]   The project [my extension name]:1.0.0-SNAPSHOT (C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpamg4yyt_\pom.xml) has 5 errors
[ERROR]     Child module C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpamg4yyt_\[my extension name]\pom.xml of C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpamg4yyt_\pom.xml does not exist
[ERROR]     Child module C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpamg4yyt_\[my extension name].channel.bin\pom.xml of C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpamg4yyt_\pom.xml does not exist
[ERROR]     Child module C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpamg4yyt_\[my extension name features]\pom.xml of C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpamg4yyt_\pom.xml does not exist
[ERROR]     Child module C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpamg4yyt_\[my extension name update]\pom.xml of C:\Users\[MY USERNAME]\AppData\Local\Temp\tmpamg4yyt_\pom.xml does not exist
[ERROR]     Unresolveable build extension: Plugin org.eclipse.tycho:tycho-maven-plugin:2.7.5 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.eclipse.tycho:tycho-maven-plugin:jar:2.7.5: Could not transfer artifact org.eclipse.tycho:tycho-maven-plugin:pom:2.7.5 from/to central (https://repo.maven.apache.org/maven2): Transfer failed for https://repo.maven.apache.org/maven2/org/eclipse/tycho/tycho-maven-plugin/2.7.5/tycho-maven-plugin-2.7.5.pom: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target -> [Help 2]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
[ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/PluginManagerException
Traceback (most recent call last):
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\build_python_extension.py", line 524, in <module>
    build_extension(
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\build_python_extension.py", line 72, in build_extension
    _build_extension(
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\build_python_extension.py", line 134, in _build_extension
    _run_maven_build(
  File "C:\Users\[MY USERNAME]\Miniconda3\envs\knime_5.1.0_extension_bundling\bin\build_python_extension.py", line 403, in _run_maven_build
    raise RuntimeError("Maven build failed.")
RuntimeError: Maven build failed.

Judging from the logs, I suspect these two URLs to be the culprits, which I can access again via my browser just like the others before:

Please let me know if my suspicion is correct that this is likely due to some sort of connection error again, so I can proceed to request for them to be unblocked.

steffen_KNIME · September 4, 2023, 8:45am

@gab1one
Hi Gabriel, have you seen this error before?

Steffen

nemad · September 4, 2023, 8:53am

Hello @DerMaxdorfer,

sorry for the delay.
I fell more like crying to be honest
It might be yet another proxy issue, this time with maven.
According to the documentation you can set up your maven settings to handle proxies as well:
https://maven.apache.org/guides/mini/guide-proxies.html

I hope this helps,
Adrian

DerMaxdorfer · September 4, 2023, 1:54pm

I feel you, @nemad. It was a walk in the park running it on my private laptop, but it can be quite painful getting it to work in a corporate network.

I’ve now requested for Apache Maven to be unblocked. Unfortunately, this usually takes between 1 and 2 weeks… I’ll let you guys know as soon as I’ve got any news.

DerMaxdorfer · September 14, 2023, 8:19am

I managed to bundle my extension, @nemad! It was once again due to an SSL certificate error. Thanks for your valuable advice along the way!

Although bundled now, I still can’t get my extension to work… But I’m going to open a new thread for that to not go too far off-topic here.

nemad · September 17, 2023, 1:14pm

Uff that was quite the hurdle!
I am glad that it finally worked and hope that we can also help you in the other thread.

Cheers,
Adrian

system · September 24, 2023, 1:14pm

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.