Issue with RDKit in Python nodes

Hi All,

today KNIME had an update for ne RDKit nodes and I installed it. Afterwards when running rdkit in python nodes I go an warning about different versions of pickling. (8.0 vs 9.0) but in fact the nodes worked and generated as far as i could tell valid output. Still it flooded the KNIME console with warnings.

I then checked and saw that the update inlcuded a new RDKIT version. So I went ahead and updated rdkit in python as well. And know when running rdkit in python nodes I get below error:

ERROR PythonKernel ImportError: DLL load failed: The specified module could not be found.
ERROR PythonKernel
ERROR PythonKernel During handling of the above exception, another exception occurred:
ERROR PythonKernel
ERROR PythonKernel ImportError: Error while loading python type extension RDKitMolDeserializer
ERROR PythonKernel Cause: DLL load failed: The specified module could not be found.
ERROR Python Script (1⇒1) 0:648:643 Execute failed: (“EOFException”): null

How can I solve this issue?

EDIT:

This is windows 64-bit and with anaconda python using root env.

Hi, there are a lot of moving parts here.
Let’s start with a couple questions:

  • which version of conda python and which RDKit version are you using?
  • can you run the RDKit itself from python? Does from rdkit import Chem work?

-greg

Hi Greg, you are right. upgrading the rdkit to newest version on conda botched something,

from rdkit import rdBase
from rdkit import Chem
from rdkit.Chem import AllChem
from rdkit.Chem import Draw

  1 try:
----> 2   from .rdBase import rdkitVersion as __version__
  3 except ImportError:
  4   __version__ = 'Unknown'
  5   raise

ImportError: DLL load failed: The specified module could not be found. 

( it still worked yesterday but obviously restarting the notebook was needed to trigger the issue after upgrade.)

EDIT:

Downgrading back to rdkit 2017_09_2 makes it work again but then I knime I get the pickling warning:

ERROR PythonKernel                    [06:47:00] Depickling from a version number (9.0)that is higher than our version (8.0).
ERROR PythonKernel                    This probably won't work.

Downgrade to 2017_09_3 did not work. Conda requested python 3.6 and I’m on 3.5 and upgrade to 3.6 also doesn’t work.

I also installed visual c++ 2017 redistributable. Already hat the 2015 one. But issue remains.

Hmmm. Ok, let’s start with the problems in conda.

Please try:
python -v -c "from rdkit import rdBase"
This will likely generate rather a lot of output, but will help us make sure that the right files are being found.

The next question: do the RDKit nodes work for you correctly without using the Python integration?

For what it’s worth: I just verified that things still work for me on Windows10 with the most recent version of the RDKit nodes (in KNIME 3.5.3) and an anaconda python install with the most recent RDKit release (2018.03.1).

Hi Greg,

yes in RDKit KNIME nodes work fine so at this point it’s technically not really a knime issue anymore.

Towards the bottom the command throws a similar error (DLL not found):

import 'site' # <_frozen_importlib_external.SourceFileLoader object at 0x000000B
04E537F98>
Python 3.5.5 |Anaconda custom (64-bit)| (default, Apr  7 2018, 04:52:34) [MSC v.
1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
# C:\Program Files\Anaconda3\lib\site-packages\rdkit\__pycache__\__init__.cpytho
n-35.pyc matches C:\Program Files\Anaconda3\lib\site-packages\rdkit\__init__.py
# code object from 'C:\\Program Files\\Anaconda3\\lib\\site-packages\\rdkit\\__p
ycache__\\__init__.cpython-35.pyc'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 968, in _find_and_load
  File "<frozen importlib._bootstrap>", line 957, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 697, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "C:\Program Files\Anaconda3\lib\site-packages\rdkit\__init__.py", line 2,
 in <module>
	from .rdBase import rdkitVersion as __version__
  File "<frozen importlib._bootstrap>", line 968, in _find_and_load
  File "<frozen importlib._bootstrap>", line 957, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 666, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 577, in module_from_spec
  File "<frozen importlib._bootstrap_external>", line 938, in create_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
ImportError: DLL load failed: The specified module could not be found.

OS is Windows Server 2012 R2. Maybe win 10 is required?

I think you should be fine with Windows Server 2012.

I don’t see anything suspicious in the output above: it looks like the RDKit is being loaded from the correct place and that the correct python files are being imported.

The next step in tracking down the problem is to using dependency walker to track down which DLL is causing the problem. You can get dependency walker (which is free), here: http://www.dependencywalker.com/

the simple install procedure is to just extract the zip file into a convenient directory (I normally just put it directly in c:\). Once it’s installed you should run depends.exe and open the file C:\Program Files\Anaconda3\lib\site-packages\rdkit\rdBase.pyd
Depends.exe is going to show a lot of information, but somewhere in there you should find a list of the DLLs it’s looking for. One (or more) will be in red, indicating that they can’t be found. We need to know what those DLLs are.

This is the result:

With exception to the python36.dll the others are available somewhere (multiple copies). I don’t know why it want python3.6 as I am on 3.5.

image

Specific error when clicking on one of the issues:

image

Maybe the build requires python36 and then it would work?

image

Does look like an issue with the newest build and python3.5. I created a new py36 env and here it works fine:

image

That’s a strange problem. Once I have a reliable network connection I will take a look and see if I can figure out what’s going on.

Thanks again for the patience

I can confirm that there is a problem with the way the python3.5 conda package was built. I’m assembling a fixed version now.

This post to the RDKit-discuss mailing list should have the solution to this problem:
https://sourceforge.net/p/rdkit/mailman/message/36304499/

I hope this helps,
-greg

I can confirm that this solved the issue. Thanks!