Export data from KNIME for processing in Python (externaly)

@PhilTroy could you explain what you mean by one machine to another? If we are talking KNIME the best way would be (well) KNIME data tables :slight_smile:. Other than that indeed Parquet seems a good option (when solving some quirks with data and time variables). Parquet does work with:

Then if you want to transfer more complicated things you can use the Integrated Deployment. There you would have data flows that would be flexible and data, Models, Flow Variables etc. that would be encapsulated together with a KNIME workflow that you can save and re-use.

How to Move Data Science into Production

Integrated Deployment KNIME Blog Articles

Hi

Pick a format for data transfer - whatever will work for your needs. And then hide that from the average user so that they just need to specify the other endpoint of the data, i.e. the recipient node. Most likely you will need an ip address for the recipient node, and a port.

Then inside the two nodes (the transmitter and the receiver) you will need to set up communications. Most likely you will use sockets or pipes, but again it should be hidden from most users, other than giving them choices.

Once done, you will draw your first diagram as usual, but it will terminate in the transmitting node. Then in the second diagram, the receiving node will act as a source of data.

Does that help?

If not, we can arrange to do a Microsoft Teams session to discuss later on in the week.

Phil

KNIME workflows can be called via REST API but obviously need proper setup in regards to input/output and you need KNIME server for this.

So in terms of OPs case the source data could be gathered from the target machine by an API call to the knime workflow.

In essence what you suggest already exists just not fro free.

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.