ERROR: RDKitAdapterCellValueFactory Python Script: Error when deserializing RDKitMolValue

gcincilla · August 22, 2023, 9:05am

Hi everyone,

After that the interoperability between KNIME and Python has been announced starting from KNIME 4.7, I’ve started working with Python, RDKit and KNIME from KNIME AP.

I’m a beginner with this integration, so I apologise in advance if this problem is related to some basic notion that I lack.

I created a Conda environment including the python modules contained in the KNIME bundled python environment and I’ve added RDKit to it.

As you can see in the attached example workflow, if I perform any operation on a column containing molecules in RDKit type format using the Python Script node, the resulting molecules have problems and cannot be rendered with KNIME (see screenshot below). This is true even if I simply copy the input RDKit type column.

Oddingly, if I test this with KNIME 4.7, no error is reported (but the molecules are not rendered anyway), while if I run it with KNIME 5.1 the error reported below is triggered.

Am I missing something or is this a bug?

2023-08-22 10:53:46,440 : ERROR : AWT-EventQueue-0 :  : RDKitAdapterCellValueFactory : Python Script : 3:2698 : Error when deserializing RDKitMolValue
java.io.IOException: Unable to interpret RDKit Molecule: Bad pickle format: ENDMOL tag not found.
	at org.rdkit.knime.types.RDKitTypeSerializationUtils.deserializeMolCell2(RDKitTypeSerializationUtils.java:53)
	at org.rdkit.knime.types.RDKitAdapterCellValueFactory$RDKitAdapterCellReadValue.getAdapterCell(RDKitAdapterCellValueFactory.java:119)
	at org.knime.core.data.v2.value.AbstractAdapterCellValueFactory$AbstractAdapterCellReadValue.getDataCell(AbstractAdapterCellValueFactory.java:259)
	at org.knime.core.data.columnar.table.ColumnarRowIterator.next(ColumnarRowIterator.java:171)
	at org.knime.core.node.BufferedDataTable$CloseableFailProveRowIterator.next(BufferedDataTable.java:527)
	at org.knime.core.node.tableview.TableContentModel.cacheNextRow(TableContentModel.java:1170)
	at org.knime.core.util.ThreadUtils.getAsBooleanWithContext(ThreadUtils.java:651)
	at org.knime.core.node.tableview.TableContentModel.cacheNextRowWithContext(TableContentModel.java:1154)
	at org.knime.core.node.tableview.TableContentModel.getRow(TableContentModel.java:1129)
	at org.knime.core.node.tableview.TableContentModel.isHiLit(TableContentModel.java:953)
	at org.knime.core.node.tableview.TableRowHeaderModel.isHiLit(TableRowHeaderModel.java:292)
	at org.knime.core.node.tableview.TableRowHeaderView.prepareRenderer(TableRowHeaderView.java:211)
	at java.desktop/javax.swing.plaf.basic.BasicTableUI.paintCell(Unknown Source)
	at java.desktop/javax.swing.plaf.basic.BasicTableUI.paintCells(Unknown Source)
	at java.desktop/javax.swing.plaf.basic.BasicTableUI.paint(Unknown Source)
	at java.desktop/javax.swing.plaf.ComponentUI.update(Unknown Source)
	at java.desktop/javax.swing.JComponent.paintComponent(Unknown Source)
	at java.desktop/javax.swing.JComponent.paint(Unknown Source)
	at java.desktop/javax.swing.JComponent.paintChildren(Unknown Source)
	at java.desktop/javax.swing.JComponent.paint(Unknown Source)
	at java.desktop/javax.swing.JViewport.paint(Unknown Source)
	at java.desktop/javax.swing.JComponent.paintChildren(Unknown Source)
	at java.desktop/javax.swing.JComponent.paint(Unknown Source)
	at java.desktop/javax.swing.JComponent.paintChildren(Unknown Source)
	at java.desktop/javax.swing.JComponent.paint(Unknown Source)
	at java.desktop/javax.swing.JComponent.paintChildren(Unknown Source)
	at java.desktop/javax.swing.JComponent.paint(Unknown Source)
	at java.desktop/javax.swing.JComponent.paintChildren(Unknown Source)
	at java.desktop/javax.swing.JComponent.paint(Unknown Source)
	at java.desktop/javax.swing.JComponent.paintChildren(Unknown Source)
	at java.desktop/javax.swing.JComponent.paint(Unknown Source)
	at java.desktop/javax.swing.JLayeredPane.paint(Unknown Source)
	at java.desktop/javax.swing.JComponent.paintChildren(Unknown Source)
	at java.desktop/javax.swing.JComponent.paintToOffscreen(Unknown Source)
	at java.desktop/javax.swing.RepaintManager$PaintManager.paintDoubleBufferedImpl(Unknown Source)
	at java.desktop/javax.swing.RepaintManager$PaintManager.paintDoubleBuffered(Unknown Source)
	at java.desktop/javax.swing.RepaintManager$PaintManager.paint(Unknown Source)
	at java.desktop/javax.swing.RepaintManager.paint(Unknown Source)
	at java.desktop/javax.swing.JComponent.paint(Unknown Source)
	at java.desktop/java.awt.GraphicsCallback$PaintCallback.run(Unknown Source)
	at java.desktop/sun.awt.SunGraphicsCallback.runOneComponent(Unknown Source)
	at java.desktop/sun.awt.SunGraphicsCallback.runComponents(Unknown Source)
	at java.desktop/java.awt.Container.paint(Unknown Source)
	at java.desktop/java.awt.Window.paint(Unknown Source)
	at java.desktop/javax.swing.RepaintManager$4.run(Unknown Source)
	at java.desktop/javax.swing.RepaintManager$4.run(Unknown Source)
	at java.base/java.security.AccessController.doPrivileged(Unknown Source)
	at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
	at java.desktop/javax.swing.RepaintManager.paintDirtyRegions(Unknown Source)
	at java.desktop/javax.swing.RepaintManager.paintDirtyRegions(Unknown Source)
	at java.desktop/javax.swing.RepaintManager.prePaintDirtyRegions(Unknown Source)
	at java.desktop/javax.swing.RepaintManager$ProcessingRunnable.run(Unknown Source)
	at java.desktop/java.awt.event.InvocationEvent.dispatch(Unknown Source)
	at java.desktop/java.awt.EventQueue.dispatchEventImpl(Unknown Source)
	at java.desktop/java.awt.EventQueue$4.run(Unknown Source)
	at java.desktop/java.awt.EventQueue$4.run(Unknown Source)
	at java.base/java.security.AccessController.doPrivileged(Unknown Source)
	at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
	at java.desktop/java.awt.EventQueue.dispatchEvent(Unknown Source)
	at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
	at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
	at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
	at java.desktop/java.awt.EventDispatchThread.pumpEvents(Unknown Source)
	at java.desktop/java.awt.EventDispatchThread.pumpEvents(Unknown Source)
	at java.desktop/java.awt.EventDispatchThread.run(Unknown Source)

python-rdkit_rendering_problem.knwf (40.6 KB)

Vernalis · September 4, 2023, 3:06pm

I don’t know, but I seem to recall hearing in discussions at the Spring Summit in Berlin that this sort of behavior which looks to be related to the (de)serialisation (Java-speak) / pickling (Python-speak) can be cause by a different version of RDKit in the Java binaries and in the Python environment. Is it possible that either of those versions change between your 2 KNIME installations?

@manuelschwarze might be able to provide more clues.

Steve

manuelschwarze · September 4, 2023, 4:37pm

Dear @gcincilla,

Steve is right. You need to ensure that the underlying RDKit binaries in KNIME RDKit Nodes and in the Python environment that is used, are the same. Best is to have them exactly the same. But normally, it would even be enough if they have the same major version, e.g. 2022_09. 2022_09_01 should be also compatible usually with 2022_09_03, for instance. Nevertheless, we had recently a situation when even a minor bugfix update in RDKit binaries broke compatibility of molecule serialization/deserialization. In order to get it working, please see that you install the correct Python RDKit version.
How do you find out which RDKit version KNIME is using? Go to KNIME Preferences, then to RDKit Nodes, and there you find the information:

Ensure that the same version is used also in your Python conda environment. You may have to upgrade or downgrade the installed version.

Kind regards,
Manuel

gcincilla · September 5, 2023, 8:08am

Thank you very much Steve @Vernalis and @manuelschwarze for the detailed explanation!
This is a very useful information and I hope it can be of help also to other users.
Soon I will try to solve this problem following your suggestions and I’ll update this thread accordingly.
Best,
Gio

manuelschwarze · September 5, 2023, 8:57am

I updated my last comment with a much better and easier way of finding out the RDKit binaries version used in the RDKit Nodes. It had slipped my mind when I created the comment… Please re-check.

Kind regards,
Manuel

gcincilla · September 6, 2023, 8:05am

I can confirm that the solution suggested by @manuelschwarze solves the problem.
Thank you very much!

P.S.: I don’t know why but it seems I cannot mark that answer as the solution.