I’m trying to load a pretrained model using the pytorch function ‘load.model’ which gives me the following error:
Can’t get attribute ‘modelname’ on <module ‘main’ from ‘C:\Program Files\KNIME\plugins\org.knime.python2_4.1.0.v201911191126\py\PythonKernelLauncher.py’> Traceback (most recent call last): File “”, line 72, in File “C:\Users\Mueller\AppData\Local\Continuum\anaconda3\lib\site-packages\torch\serialization.py”, line 529, in load return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args) File “C:\Users\Mueller\AppData\Local\Continuum\anaconda3\lib\site-packages\torch\serialization.py”, line 702, in _legacy_load result = unpickler.load() AttributeError: Can’t get attribute ‘modelname’ on <module ‘main’ from ‘C:\Program Files\KNIME\plugins\org.knime.python2_4.1.0.v201911191126\py\PythonKernelLauncher.py’>
The script is running without any problems in spyder on the same machine.
Is it possible to use pretrained Pytorch models using KNIME at all?
Are there any restrictions using Python in KNIME?
I am using KNIME 4.1.1, Pytorch 1.4 and latest Python on Windows 10 Pro 1903.
Would you mind sharing the script (or the entire KNIME workflow) you are trying to run? Which Python node are you using to execute the script?
In general, it should be possible to execute (almost) arbitrary Python scripts in KNIME as long as they fulfill some node-specific requirements (e.g., the Python Script (1⇒1) node requires the script to define a variable output_table that holds a pandas DataFrame that can be transferred to KNIME, etc.).
I believe the problem is caused by the fact that KNIME’s Python nodes do not execute the user-provided script directly as the Python main module but via Python’s eval function.
Your saved PyTorch model references the class Modelname. At the time of saving, Modelname was part of the main module of the script that was saving the model. Therefore, upon loading, Modelname is tried to be obtained from main again. Since your script is not main in KNIME, the definition of Modelname you provide is not found.
I do not have any deeper understanding of using PyTorch, unfortunately, but looking at their documentation suggests that saving/loading the model’s state_dict instead of the entire model is more portable. The documentation states:
The disadvantage of this [= saving/loading the entire model] approach is that the serialized data is bound to the specific classes and the exact directory structure used when the model is saved. The reason for this is because pickle does not save the model class itself. Rather, it saves a path to the file containing the class, which is used during load time. Because of this, your code can break in various ways when used in other projects or after refactors.
Could you give the state_dict approach a try and see if it works?
Alternatively, you could move Modelname into its own Python module and import it from there when constructing, and later loading, the model. This way, the saved reference to the class will stay valid. You would need to make sure that the module is always on the module search path, however.
