Bert GPU problem


I was using the Redfield Bert extensions and they were working fine until I installed the new Redfield Bert Nodes extensions, which creates some sort of default to use GPU (bundled config). I did want to use my GPU as I have done before in KNIME with Keras nodes (I did configure it CUDA V10.0.130). However now I am getting all sort of errors and I am unable to use the “bert model selector.” I also tried 1) using miniconda, with other enviroments that were working previously, and 2) using the option of creating an enviroment in the Bert Nodes tab on Preferences. Nothing seems to work. These are the errors that I am getting:

1)Keeps loading forever and doesn’t report error

2)ERROR BERT Model Selector 3:171 Failed to initialize LZ4 libraries. The Columnar Table Backend won’t work properly

3)Execute failed: no jnilz4 in java.library.path: C:\Program Files\KNIME;C:\WINDOWS\Sun\Java\bin;C:\WINDOWS\system32;C:\WINDOWS;C:/Program Files/KNIME//plugins/org.knime.binary.jre.win32.x86_64_17.0.5.20221116/jre/bin/server;C:/Program Files/KNIME//plugins/org.knime.binary.jre.win32.x86_64_17.0.5.20221116/jre/bin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0\bin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0\libnvvp;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Windows\System32\OpenSSH;C:\Program Files\dotnet;C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64;C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\Scripts;C:\Program Files (x86)\Calibre2;C:\Program Files\LIWC-22;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0;C:\WINDOWS\System32\OpenSSH;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Users\Windows\AppData\Local\Microsoft\WindowsApps;C:\Users\Windows.dotnet\tools;;C:\Program Files\KNIME;;.

4)Execute failed: Could not initialize class org.knime.python3.scripting.Python3KernelBackend

Any recommendation is welcome! thanks

Hello again,

Just an update. I uninstalled the redfield bert nodes extension and installed again.

First I tried using with the bundled installation of the redfield bert nodes, but I got this error when trying to run the “bert model selector” Execute failed: Could not initialize class org.knime.python3.scripting.Python3KernelBackend

Second, I tried creating a bert enviroment in the preferences section of Bert (redfield_bert), but it did not work either. I got the same error: Execute failed: Could not initialize class org.knime.python3.scripting.Python3KernelBackend

As said before my laptop has cuda v.10 installed properly as I have used it in KNIME before.

Please if I could get some support, thanks!

Last update. I noticed the problem had to with Python not starting, and that led me to a solution in the forum asking to check if the “KNIME python integration - Columnar table back end” . It was not installed so I installed it, but still nothing works. I have tried a bunch of things but nothing. The Bert model selector gives me an errors or it starts running forever. This is the last log in case it can help:

2023-05-20 23:36:51,750 : ERROR : KNIME-Worker-32-BERT Model Selector 3:888 : : Node : BERT Model Selector : 3:888 : Execute failed: An error occurred while trying to launch Python: An exception occured while running the Python kernel. See log for details.
org.knime.dl.core.DLInvalidEnvironmentException: An error occurred while trying to launch Python: An exception occured while running the Python kernel. See log for details.
at se.redfield.bert.core.BertCommands.createKernel(
at se.redfield.bert.core.BertCommands.(
at se.redfield.bert.nodes.selector.BertModelSelectorNodeModel.downloadOrCheckModel(
at se.redfield.bert.nodes.selector.BertModelSelectorNodeModel.execute(
at org.knime.core.node.NodeModel.executeModel(
at org.knime.core.node.Node.invokeFullyNodeModelExecute(
at org.knime.core.node.Node.execute(
at org.knime.core.node.workflow.NativeNodeContainer.performExecuteNode(
at org.knime.core.node.exec.LocalNodeExecutionJob.mainExecute(
at org.knime.core.node.workflow.NodeExecutionJob.internalRun(
at org.knime.core.util.ThreadUtils$RunnableWithContextImpl.runWithContext(
at org.knime.core.util.ThreadUtils$
at java.base/java.util.concurrent.Executors$ Source)
at java.base/ Source)
at org.knime.core.util.ThreadPool$
at org.knime.core.util.ThreadPool$
Caused by: org.knime.python2.kernel.PythonIOException: An exception occured while running the Python kernel. See log for details.
at org.knime.python2.kernel.PythonKernelQueue$KeyedPooledPythonKernelFactory.createKernel(
at org.knime.python2.kernel.PythonKernelQueue$KeyedPooledPythonKernelFactory.populateHolder(
at org.knime.python2.kernel.PythonKernelQueue$KeyedPooledPythonKernelFactory.passivateObject(
at org.knime.python2.kernel.PythonKernelQueue$KeyedPooledPythonKernelFactory.passivateObject(
at org.apache.commons.pool2.impl.GenericKeyedObjectPool.returnObject(
at org.knime.python2.kernel.PythonKernelQueue.lambda$1(
at java.base/ Source)
Caused by: Could not connect to the Python process.
at org.knime.python3.DefaultPythonGateway.waitForConnection(
at org.knime.python3.DefaultPythonGateway.(
at org.knime.python3.DefaultPythonGateway.create(
at org.knime.python3.scripting.Python3KernelBackend.(
at org.knime.python3.scripting.Python3KernelBackend.(
at org.knime.python3.scripting.Python3KernelBackendFactory.createBackend(
at org.knime.python2.kernel.PythonKernelQueue$KeyedPooledPythonKernelFactory.createKernel(
… 6 more

Hello @anonymous_mkt

Thank you for your feedback and detailed stacktraces. I see that you have updated CUDA to version 10.0, so I would expect problems here. Since it is recommended to go with CUDA 10.1, cudNN 7.6 and TF 2.2.0.

This issue might require investigation, so it is not going to be a quick fix. The only suggestion for you that I have is that you can try to follow the instructions written in this blog post:

In the meantime could you please provide the information about:

  • OS and version;
  • CUDA and cudNN;
  • Python;
  • BERT and TF packages versions;
  • Nvidia driver version.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.