The model writer writes objects in a KNIME specific data format meant to be read by Model Reader nodes. The python object reader / writer nodes were created to allow writing / reading of data in a format that can be understood by external python processes.
Take a look at this workflow for a workaround if you can’t use these nodes yet:
Python + KNIME expert @mlauber71 explains there how to read + write data inside python scripting nodes.
@JaeHwanChoi the model writer will zip a given model to store it. Underneath there has to be a model that the package that you used can read(KNIME ML to Python - #2 by mlauber71 ). If you want to use it with Jupyter notebooks this has to work in both environments.
You could try and store a model just as pickel and use knime as interface for some python models. Here are examples from several versions of knime and python:
There are other frameworks you could use. Knime started to generically support Sklearn for certain models:
If you want to interact between knime and python maybe the H2O.ai framework is for you. You can interchange models via MOJO files.
You might derive further inspiration from this collection:
What I ultimately want to do is to save the information of an ML model generated by the Pickeld Object output port of the “Python Script” node in KNIME to a specific repository with the “Model Writer” node.
After that, I want to load the model extension in the repository in Jupyter and this is where I am getting the error.
I can’t use H2O Mojo, because the analysis must use only algorithms that I have written within the Python Script.
I really need to read the files derived by Model Writer in Jupyter, so I would like to find a way around this.
If you really want to read the output of the “Model Writer” node you need to find out the exact binary output format written by the node and re-implement the de-serialization in Python. You can look up the code of the “Model Writer” node here and the Python port object here.
However, I don’t think that this is a good approach. Instead, I recommend using a “Python Script” node as the model writer. Use a little script that pickles your model and directly writes it to a file on the disk.
So it looks like I have to follow a bad approach to interact with the “Model Writer” node.
I’m currently in the middle of a project, so it’s very difficult to look at the github you shared in detail.
That said, I don’t see a solution other than exporting to your local PC or a specific disk space as code within a Python Script.
This would work on a personal PC to save to a personal disk, but if the workflow is running on KNIME Server and the repository where the models are stored requires an account and password to access, such as MinIO, it is not feasible to apply it as code within Python Script. (MinIO has a Python package, so I need to test if it works when running on KNIME Server.)
I need to go through this similar process with Pyspark, which also seems very difficult.
I’d be grateful to hear if I’m wrong about any of the above.
Maybe you can provide a sample workflow or a screenshot of your task so we better understand the challenge. What part will be in knime and what part in Python. If you use knime server you will have to make sure the Python on the server contains the same packages and setup as the desktop version.
If you want to use PMML that is also an option but you will be limited to models that would support this format.