DB Insert node with optional data output port

Hello there,

is it possible to have option in DB Insert (and guess same applies for DB Writer) node to make data output port optional? I’m transferring huge amount of data from one database to another in a loop and don’t need data in KNIME. Reading data into KNIME can last a lot for big and wide tables which makes workflow unnecessary slow…

Have a nice day,
Ivan

Hello @ipazin :slight_smile: ,

May I ask why you don’t use the DB SQL Executor node?

You can extract the SQL query of what you have already applied on the source table(s) using the DB Query Extractor node (which is exactly what you pass to DB Insert/Writer) and then use the flow variable in the DB SQL Executor. For example to create a new table:

INSERT INTO 'new_table' $${Ssql}$$;

Where $${Ssql}$$ is the extracted query from DB Query Extractor.

Screenshot 2024-08-01 at 10.21.01

1 Like

Hello @armingrudd,

we are talking about two different databases so not sure I can apply your idea. Additionally consider use case where you read data from non-DB data source into KNIME, apply transformation on it and then write into DB. Usually you don’t need this data later on in KNIME.

Br,
Ivan

1 Like

You are right, I thought you were dealing with tables in the same database.

So, if you have no option than having the input data in AP, then ouput data port of the node is an exact reference of the input table + optional status column. So, if you uncheck the status column then nothing new is calculated. It actually doesn’t make any difference if you had the output data port of the insert node disabled. The “data.xml” under DB Insert only adds this:
<entry key="table_reference_ID" type="xint" value="<table ID>"/>

It’s not reading data from DB to AP, it’s just a representation of an existing table.

Does it make sense or am I missing something?

1 Like

Hello @armingrudd,

I see. Then not inserting status column should actually do what I’m looking for. Will give it a try and let you know.

Br,
Ivan

1 Like

Hello @armingrudd,

checked it and this makes a difference. Maybe worth adding to node description that adding two additional columns can impact performance or something similar. Thank you!

Br,
Ivan

1 Like