Python Node Extension deployment failing on Apple Macs: Could not connect to the Python process

Hi

We have recently created a custom Python Node extension, and are running into some problems trying to deploy it to Apple Macs.
The deployment bundle has being successfully deployed on 3 different Windows machines, but it has failed on both the MacBook laptops we have tried.

We get a similar error on both Macs: Could not connect to the Python process
We are assuming that this indicates a problem with the bundled Conda environment, but please let us know if that is not the case.
Do you know the cause of/solution to this problem?
Can you also let us know where the deployed Conda instance is installed on a Mac, so we can test the deployment environment directly.

For reference, here are some details of the two Mac computers where the installation failed, and the errors printout from knime.log:

  1. MacBook Pro used to successfully develop the Python Node Extension and create the deployment bundle
    1. macOS Monterey Version 12.6
    2. MBP 15-inch 2016
    3. Intel Core i7
    4. KNIME V4.6.1
    5. KNIME Python Extension Development (Labs) V4.6.2.v202209150850
  2. MacBook Air - never used for development, with a new installation of KNIME 4.6.2
    1. macOS BigSur V 11.6
    2. MacBook Air M1 2020

Errors:

2022-09-26 17:13:42,869 : ERROR : ForkJoinPool.commonPool-worker-1 : : PurePythonNodeSetFactory : : : java.net.ConnectException: Could not connect to the Python process.
2022-09-26 17:13:42,869 : ERROR : ForkJoinPool.commonPool-worker-1 : : PurePythonNodeSetFactory : : : Failed to parse Python node extension at path ‘/Applications/KNIME 4.5.0.app/Contents/Eclipse/plugins/com.bctpartners.tree_ruleset_extension_1.0.0.202209190121/src/main/python’.

Thanks
Patrick

Hi @poconnell,
Welcome to the KNIME Forum!

Sadly the error message doesn’t contain enough information. There should be more information about why the extension couldn’t be parsed in the “DEBUG” log (Enable here: File->Preferences->KNIME->Log File Log Level. Open here: View->Open KNIME Log).

It can be a bit hard to find the important details in the cluttered DEBUG log. In the latest nightly we added more information to the ERROR log if the parsing of an extension failed. It would be valuable feedback if you could try the nightly build and report if this information helped.

The bundled environment should be located in the env folder in a plugin called <extension_name>.channel.bin.macos.x86_64_<version>. The plugins are located in the folder <KNIME.app>/Contents/Eclipse/plugins/.
For the machine which produced the error message this folder should be:

/Applications/KNIME 4.5.0.app/Contents/Eclipse/plugins/com.bctpartners.tree_ruleset_extension.channel.bin.macos.x86_64_1.0.0.202209190121/env

You can just run the bin/python executable to test the environment.

2 Likes

OK, thanks for the quick reply. Let me try that.

We have tried all three suggestions:

  1. DEBUG mode was turned on for the log. This didn’t really provide any more insight into the problem. The reason the Python Extension can’t be parsed is because KNIME can’t connect to the Python process. But why it can’t connect to the Python process is still unclear.
  2. The nightly build was installed, but again this didn’t identify any additional information for this particular problem.
  3. Finally, the bundled Python environment was tested, but this seemed to work fine.

We still need some help in debugging the problem here.
Maybe listing more details describing its attempts to connect to the Python process in the debug log?

Here are some more technical details of what was tried.

For testing the Python environment we were able to execute automated tests included in the Extension from the bundled Python environment:

Patrick@Cs-MacBook-Air bin % pwd
/Applications/KNIME 4.7.0_2022-09-30_14-04-19.app/Contents/Eclipse/plugins/com.bctpartners.tree_ruleset_extension.channel.bin.macosx.x86_64_1.0.0.202209190121/env/bin
Patrick@Cs-MacBook-Air bin % ./python -m pytest “/Applications/KNIME 4.7.0_2022-09-30_14-04-19.app/Contents/Eclipse/plugins/com.bctpartners.tree_ruleset_extension_1.0.0.202209190121/src/main/python/get_rules_from_pmml_test.py”
======= test session starts =======
platform darwin – Python 3.9.13, pytest-7.1.2, pluggy-1.0.0
rootdir: /Applications/KNIME 4.7.0_2022-09-30_14-04-19.app/Contents/Eclipse/plugins
plugins: cov-3.0.0
collected 6 items

…/…/…/com.bctpartners.tree_ruleset_extension_1.0.0.202209190121/src/main/python/get_rules_from_pmml_test.py … [100%]

========= 6 passed in 0.37s ==============

This seems to indicate the bundled Python environment is OK.

These are the relevant lines from the Nightly Build log file in DEBUG mode:

2022-10-14 16:50:09,727 : ERROR : Worker-2: Node Repository Loader : : PurePythonNodeSetFactory : : : java.net.ConnectException: Could not connect to the Python process.
2022-10-14 16:50:09,729 : ERROR : Worker-2: Node Repository Loader : : PurePythonNodeSetFactory : : : Failed to parse Python node extension at path ‘/Applications/KNIME 4.7.0_2022-09-30_14-04-19.app/Contents/Eclipse/plugins/com.bctpartners.tree_ruleset_extension_1.0.0.202209190121/src/main/python’.

We can send the entire log file, if that is helpful.
Thanks
Patrick

Hi KNIME team

Any feedback on this?
Our extension deploys correctly to PCs but not on Macs.

As described above we have tried all the KNIME team’s suggestions: testing the bundled Python environment, turning on Debug logging, and using the nightly build.
The only message is “Could not connect to the Python process” which is not enough to debug this problem.

We can provide the entire log file, if that would be helpful. (It wasn’t immediately apparent how to attach a file to a forum post.)
We can also provide the extension itself, if that will help identify where the KNIME Python Development Labs extension’s error handling or debugging needs to be extended to provide sufficient useful information to allow developers to debug Mac installation issues with their custom extensions.

Thanks
Patrick