''My Template Node'' node does not show up according Create a New Python based KNIME Extension

I try to create my first pure python extension following the guide " Create a New Python based KNIME Extension" (Create a New Python based KNIME Extension). In step 8 the “My Template Node” does not appear in the node repository. Therefore, some questions arises:

(1) During installation via “conda install python=3.9 knime-python-base=4.7 knime-extension=4.7 -c knime -c conda-forge” in the conda environment a request for administrative previleges came up. I choosed ‘No’ (Windows OS) which is shown in the command line print out:

Downloading and Extracting Packages

Preparing transaction: done
Verifying transaction: done
Executing transaction: \ menuinst called from non-root env C:\Program Files\User\miniconda\envs\KNIME
done

Is this an issue?
(2) My KNIME Version is 5.1, but the knime-python-base and knime-extension are pinned to 4.7 - could that be an issue?
(3) Regarding config.yml and knime.yml:
(3a) Is it OK to use forward slashes only (even on Windows)?
(3b) Must spaces in path names somehow escaped (e.g. c:/Program Files/)?
(4) Regarding knime.yml:
(4a) Are _ allowed in name? Or in general: Which characters are allowed/forbidden in name?
(4b) Which values are possible for group_id? (I did not changed that value in the template, but it is interesting to know.
(5) Regarding adding *-Dknime.python.extension.config=C:/Program Files/User/dev/python_based_knime_ext/basic/config.yml

  • to knime.ini:
    (5a) Must spaces soomehow escaped?
    (5b) Could this line be placed anywhere or in a specific section of the knime.ini?

I hope my questions help to improve the " Create a New Python based KNIME Extension" section.

Hi @spf,

nice that you try it out!

The implicit question first: why does your node not appear in the node repository?
Please have a look at the Troubleshooting section of the guide, the first step there would be to examine the KNIME Log.

(1) Well, can you go into your environment (via conda activate your_environment) and see if the packages knime-python-base and knime-extension are in that environment (via conda list)? That should answer your question. But I would assume that this is an issue.
(2) Please pin versions to the KNIME Analytics Platform version (i.e. to 5.1 instead of 4.7)
(3a) No, only in the knime.ini when configuring the line -Dknime.python.extension.config=path/to/your/config.yml you need to always use forward slashes; everywhere else you have to comply to how it is written in your OS (i.e. on Windows the config.yml can contain lines like src: C:\Users\knime\git\my_extension
(3b) I assume that spaces must be escapes (C:/Program\ Files/), but I am not certain. Please check the KNIME Log to see whether that was the issue.
(4) I do not see which characters (except \ and / of course) should be forbidden, but I also would not use any other special characters except . and _. This holds for both name and group_id.
(5a) See (3b).
(5b) Anywhere, but I usually put it at the end.

Please let me know what the outcome of your KNIME Log examination is.

Best regards
Steffen

Hi Steffen,

thanks for your reply.

to (1) conda list shows that both packages are installed properly.
to (2) this needs to be fixed in the desciption. My upgrade to 5.1 was successful for both packages.

I could not found the menu entry to show the log in new UI (unfinished development!). However, switching to the old give me:

2023-12-12 14:56:25,245 : ERROR : ForkJoinPool.commonPool-worker-2 :  : NodeSetFactoryExtension :  :  : NodeSetFactory 'org.knime.python3.nodes.PurePythonNodeSetFactory' from plugin 'org.knime.python3.nodes' could not be created.
org.eclipse.core.runtime.CoreException: Plug-in "org.knime.python3.nodes" was unable to instantiate class "org.knime.python3.nodes.PurePythonNodeSetFactory".
	at org.eclipse.core.internal.registry.osgi.RegistryStrategyOSGI.throwException(RegistryStrategyOSGI.java:212)
	at org.eclipse.core.internal.registry.osgi.RegistryStrategyOSGI.createExecutableExtension(RegistryStrategyOSGI.java:206)
	at org.eclipse.core.internal.registry.ExtensionRegistry.createExecutableExtension(ExtensionRegistry.java:920)
	at org.eclipse.core.internal.registry.ConfigurationElement.createExecutableExtension(ConfigurationElement.java:246)
	at org.eclipse.core.internal.registry.ConfigurationElementHandle.createExecutableExtension(ConfigurationElementHandle.java:63)
	at org.knime.core.node.extension.NodeSetFactoryExtension.from(NodeSetFactoryExtension.java:265)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(Unknown Source)
	at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Unknown Source)
	at java.base/java.util.stream.ReferencePipeline$Head.forEach(Unknown Source)
	at java.base/java.util.stream.ReferencePipeline$7$1.accept(Unknown Source)
	at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Unknown Source)
	at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
	at java.base/java.util.stream.ReduceOps$ReduceTask.doLeaf(Unknown Source)
	at java.base/java.util.stream.ReduceOps$ReduceTask.doLeaf(Unknown Source)
	at java.base/java.util.stream.AbstractTask.compute(Unknown Source)
	at java.base/java.util.concurrent.CountedCompleter.exec(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinPool.scan(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source)
Caused by: java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Unknown Source)
	at java.base/java.lang.reflect.Constructor.newInstance(Unknown Source)
	at org.eclipse.core.internal.registry.osgi.RegistryStrategyOSGI.createExecutableExtension(RegistryStrategyOSGI.java:204)
	... 20 more
Caused by: while parsing a block mapping
 in 'reader', line 1, column 1:
    org.tutorial.first_extension: or ... 
    ^
expected <block end>, but found '<block mapping start>'
 in 'reader', line 2, column 3:
      src: C:\Program Files\User\dev\p ... 
      ^

	at org.yaml.snakeyaml.parser.ParserImpl$ParseBlockMappingKey.produce(ParserImpl.java:572)
	at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:158)
	at org.yaml.snakeyaml.parser.ParserImpl.checkEvent(ParserImpl.java:148)
	at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java:214)
	at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:144)
	at org.yaml.snakeyaml.composer.Composer.getNode(Composer.java:85)
	at org.yaml.snakeyaml.composer.Composer.getSingleNode(Composer.java:108)
	at org.yaml.snakeyaml.constructor.BaseConstructor.getSingleData(BaseConstructor.java:139)
	at org.yaml.snakeyaml.Yaml.loadFromReader(Yaml.java:524)
	at org.yaml.snakeyaml.Yaml.load(Yaml.java:452)
	at org.knime.python3.nodes.PythonExtensionPreferences.parseYmlFile(PythonExtensionPreferences.java:123)
	at org.knime.python3.nodes.PythonExtensionPreferences.loadConfigs(PythonExtensionPreferences.java:112)
	at org.knime.python3.nodes.PythonExtensionPreferences.getPathsToCustomExtensions(PythonExtensionPreferences.java:84)
	at org.knime.python3.nodes.PurePythonNodeSetFactory.getExtensionsFromPreferences(PurePythonNodeSetFactory.java:121)
	at org.knime.python3.nodes.PurePythonNodeSetFactory.parseExtensions(PurePythonNodeSetFactory.java:112)
	at org.knime.python3.nodes.extension.ExtensionNodeSetFactory.<init>(ExtensionNodeSetFactory.java:112)
	at org.knime.python3.nodes.PurePythonNodeSetFactory.<init>(PurePythonNodeSetFactory.java:108)
	... 26 more

trying to escape space with backslash:

Caused by: java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Unknown Source)
	at java.base/java.lang.reflect.Constructor.newInstance(Unknown Source)
	at org.eclipse.core.internal.registry.osgi.RegistryStrategyOSGI.createExecutableExtension(RegistryStrategyOSGI.java:204)
	... 30 more
Caused by: while parsing a block mapping
 in 'reader', line 1, column 1:
    org.tutorial.first_extension: or ... 
    ^
expected <block end>, but found '<block mapping start>'
 in 'reader', line 2, column 3:
      src: C:\Program\ Files\User\dev\ ... 
      ^

Maybe the first line in config.yaml is wrong, because I only change the part behind the colon: org.tutorial.first_extension: org.tutorial.extract_validation_reports, thus first_extension is not valid anymore?

Best regards,
Sebastian

do not escape the space in config.yml reduces the log output to:

2023-12-12 15:12:59,000 : ERROR : Worker-1: Node Repository Loader :  : PythonExtensionPreferences :  :  : Failed to parse Python extension config.
2023-12-12 15:12:59,005 : ERROR : Worker-1: Node Repository Loader :  : PythonExtensionPreferences :  :  : Failed to parse Python extension config.
2023-12-12 15:12:59,005 : ERROR : Worker-1: Node Repository Loader :  : PythonExtensionPreferences :  :  : Failed to parse Python extension config.
2023-12-12 15:12:59,005 : ERROR : Worker-1: Node Repository Loader :  : PythonExtensionPreferences :  :  : Failed to parse Python extension config.
2023-12-12 15:12:59,219 : ERROR : Worker-1: Node Repository Loader :  : PythonExtensionPreferences :  :  : Failed to parse Python extension config.
2023-12-12 15:12:59,220 : ERROR : Worker-1: Node Repository Loader :  : PythonExtensionPreferences :  :  : Failed to parse Python extension config.
2023-12-12 15:12:59,220 : ERROR : Worker-1: Node Repository Loader :  : PythonExtensionPreferences :  :  : Failed to parse Python extension config.
2023-12-12 15:12:59,220 : ERROR : Worker-1: Node Repository Loader :  : PythonExtensionPreferences :  :  : Failed to parse Python extension config.
2023-12-12 15:12:59,326 : ERROR : Worker-1: Node Repository Loader :  : PythonExtensionPreferences :  :  : Failed to parse Python extension config.
2023-12-12 15:12:59,326 : ERROR : Worker-1: Node Repository Loader :  : PythonExtensionPreferences :  :  : Failed to parse Python extension config.
2023-12-12 15:12:59,327 : ERROR : Worker-1: Node Repository Loader :  : PythonExtensionPreferences :  :  : Failed to parse Python extension config.
2023-12-12 15:12:59,327 : ERROR : Worker-1: Node Repository Loader :  : PythonExtensionPreferences :  :  : Failed to parse Python extension config.

For completeness here the content of config.yml:

org.tutorial.first_extension: org.tutorial.extract_validation_reports # {group_id}.{name} from the knime.yml
src: C:\Program Files\User\dev\python_based_knime_ext\basic\tutorial_extension # Path to folder containing the extension files
conda_env_path: C:\Program Files\User\miniconda\envs\KNIME # Path to the Python environment to use
debug_mode: true # Optional line, if set to true, it will always use the latest changes of execute\configure, when that method is used within the KNIME Analytics Platform

Hi Sebastian @spf

Ah, the config.yml needs to look like this:

org.tutorial.extract_validation_reports: # {group_id}.{name} from the knime.yml
src: C:\Program Files\User\dev\python_based_knime_ext\basic\tutorial_extension # Path to folder containing the extension files
conda_env_path: C:\Program Files\User\miniconda\envs\KNIME # Path to the Python environment to use
debug_mode: true # Optional line, if set to true, it will always use the latest changes of execute\configure, when that method is used within the KNIME Analytics Platform

Does that work?

Best regards
Steffen

No.

2023-12-13 09:51:02,472 : ERROR : Worker-0: Node Repository Loader :  : PythonExtensionPreferences :  :  : Failed to parse Python extension config.
2023-12-13 09:51:02,478 : ERROR : Worker-0: Node Repository Loader :  : PythonExtensionPreferences :  :  : Failed to parse Python extension config.
2023-12-13 09:51:02,480 : ERROR : Worker-0: Node Repository Loader :  : PythonExtensionPreferences :  :  : Failed to parse Python extension config.
2023-12-13 09:51:02,480 : ERROR : Worker-0: Node Repository Loader :  : PythonExtensionPreferences :  :  : Failed to parse Python extension config.
2023-12-13 09:51:02,615 : ERROR : Worker-0: Node Repository Loader :  : PythonExtensionPreferences :  :  : Failed to parse Python extension config.
2023-12-13 09:51:02,616 : ERROR : Worker-0: Node Repository Loader :  : PythonExtensionPreferences :  :  : Failed to parse Python extension config.
2023-12-13 09:51:02,616 : ERROR : Worker-0: Node Repository Loader :  : PythonExtensionPreferences :  :  : Failed to parse Python extension config.
2023-12-13 09:51:02,616 : ERROR : Worker-0: Node Repository Loader :  : PythonExtensionPreferences :  :  : Failed to parse Python extension config.
2023-12-13 09:51:02,773 : ERROR : Worker-0: Node Repository Loader :  : PythonExtensionPreferences :  :  : Failed to parse Python extension config.
2023-12-13 09:51:02,774 : ERROR : Worker-0: Node Repository Loader :  : PythonExtensionPreferences :  :  : Failed to parse Python extension config.
2023-12-13 09:51:02,774 : ERROR : Worker-0: Node Repository Loader :  : PythonExtensionPreferences :  :  : Failed to parse Python extension config.
2023-12-13 09:51:02,774 : ERROR : Worker-0: Node Repository Loader :  : PythonExtensionPreferences :  :  : Failed to parse Python extension config.

I just copy pasted your version, the intendation was missing:

org.tutorial.extract_validation_reports: # {group_id}.{name} from the knime.yml
  src: C:\Program Files\User\dev\python_based_knime_ext\basic\tutorial_extension # Path to folder containing the extension files
  conda_env_path: C:\Program Files\User\miniconda\envs\KNIME # Path to the Python environment to use
  debug_mode: true # Optional line, if set to true, it will always use the latest changes of execute\configure, when that method is used within the KNIME Analytics Platform

Does that work?

Steffen

Interesting: I thought that I made that indentation by accident.

Note: By default my editor changes indentation created by TAB into several SPACES.

Oh, Thanks a lot. The node is finally visible!

Great to hear!

Spaces is good. It also does not matter how many spaces (we usually do 2), as long as every indented line has the same amount of spaces.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.