Running KNIME from Jupyter

[Experimenter]

Hey there.
I just found out that it is possible to run KNIME workflow from Jupyter. Even tried this (out of curiosity, actually), however I am wonderingnow on what is the status of the KNIME module for python (like, is it a serious development or just a gimmick).
I had two issues with it:

• I could not find any documentation for using the Python module (to see what are names of data exchange variables of the workflow object)
• from what I see by calling dir(wf) in Jupyter, it seems that only table-type data exchange is possible from Jupyter, no variables. Is this right?

[mlauber71]

It is a serious function. If you want to see how you could use this feature I have compiled a small example how to use Jupyter notebook (ressources) in KNIME and also how to include existing .PY files in KNIME.

Use functions and Python code from an external .PY file or a Juypter notebook...

And you can create and use flow variables in Python:

Can I create a flow variable in Python Snippet?

Hi Stoeter I am guessing this might be true if people are using python from the community extension, but in case they really need to do it, they can also use KNIME python integration to do it. Again, multiple options available within KNIME, and I guess this was part of the confusion. Here is one example how to do it using the Python Integration nodes: Cheers Ana

Also if you have problems bringing data from and to Python you could use Parquet or ORC files.

Read ORC file into KNIME's Python node – mlauber71

[Experimenter]

Thanks. However I am not sure if your reply precisely relates to what I am up to.
I am not trying to integrate Python in Knime, it is vice versa, I am integrating KNIME into Jupyter notebook. Like,

import knime

wf = knime.Workflow(
    workspace_path = r"C:\Development\knime",
    workflow_path = r"Experiments\Jupyter demo"
)

Then I can pass data to workflow:

names_list = pd.DataFrame(["George", "Georgey", "Peter", "Geology", "Anna", "Anatoliy"], columns = ["name"])
wf.data_table_inputs[0] = names_list

And then execute:

wf.execute()

So what I can’t do is passing variable to the wf object. At least I can’t find how. I can do it via table, but variable is more apt in certain situations.

Python integration nodes in KNIME work fine for me, except that they get really stuck when I try to pass over large tables. But that’s another issue.

[potts]

Hi @Experimenter –

So what I can’t do is passing variable to the wf object.

Among the feature requests tracked on the github repository for knimepy (see Issues · knime/knimepy · GitHub) is a proposal to support the setting of flow variables. If this is what you mean, it would help if you could add some encouragement to the github issue, requesting that this indeed gets implemented.

I am wonderingnow on what is the status of the KNIME module for python (like, is it a serious development or just a gimmick)

I am the primary author of the knimepy package. Sometimes it helps to read things, imagining how they might be received by someone else, say the author of a tool. If you are serious about using this tool, I would appreciate your help by requesting features (ref: issues on github) that would genuinely help you in your work. Finding other ways to contribute back is also very much welcomed.

Davin

[Experimenter]

Hey Davin, thank you for the answer. My apologies if my question touched somebody’s feelings. Just I understand that Jupyter/Python side integration was created 1,5 years ago with Knime 3.7, and it does not seem too mature 19 months later, considering that there is no documentation (or I just am really bad at googling) and from forums it seems that feature is hardly used by anybody. Really, only way I could find out that there is no property for flow variables container input, was just listing all the wf object’s parameters from python side.

I read about Jupyter integration just today in KNIME newsletter email, so I immediately went and checked how it works. Do not have a real use case for now, though. Will check the features requesting part in github.