KNIME Source to Target Data Flow

I’m new to KNIME, so this question may be trivial and already answered elsewhere. Say I have a workflow configured in KNIME for Windows that takes as source an Oracle table on one system, and as target an SQL Server table on a remote system (three systems, total). How will the source data travel from the Oracle table to the target SQL Server table? Will the data be routed directly between those two systems, or will the data travel through the Windows system on which I have KNIME installed? I appreciate your response in advance.

@kamrankns welcome to the KNIME forum. It will have to go thru the system KNIME is running on. But here is an example how to do this with streaming in order to make it more efficient.

2 Likes

Thanks for the link. This will get me partly there but the data will still travel through the system that runs KNIME. Although KNIME Streaming Execution skips landing data on the KNIME system, the cost to the network and the latency it causes will still have to be paid. I guess what I’m looking for is to use KNIME to manage the configuration of source to target data move, but then run the resulting process only on the source and target systems. So if that is possible, please let me know. Thank you again.

Hello @kamrankns,

and welcome to KNIME Community!

To my understanding to transfer data from one system to another (in this case databases) you need a connection between those two systems. In case you are using KNIME then KNIME is that connection as it connects to source system and grabs data, then connects to another system where it sends data (regardless does it write data down to system where KNIME is installed or uses streaming execution) using some protocol underneath these connections. KNIME cannot (and I’m not aware of other tools that might do this) open a direct connection between two systems/databases/locations to transfer data without interfering.

Hope this clarifies how one can use KNIME to transfer data between two databases.

Br,
Ivan

Thank you for the response. That is what I figured and was worried about. Maybe I’ll need to develop an extension node that would do what I want. Thank you again…

ks

1 Like

Hello @kamrankns,

If possible that would be great and welcomed by many :wink:

Br,
Ivan

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.