is it possible to have option in DB Insert (and guess same applies for DB Writer) node to make data output port optional? I’m transferring huge amount of data from one database to another in a loop and don’t need data in KNIME. Reading data into KNIME can last a lot for big and wide tables which makes workflow unnecessary slow…
You can extract the SQL query of what you have already applied on the source table(s) using the DB Query Extractor node (which is exactly what you pass to DB Insert/Writer) and then use the flow variable in the DB SQL Executor. For example to create a new table:
INSERT INTO 'new_table' $${Ssql}$$;
Where $${Ssql}$$ is the extracted query from DB Query Extractor.
we are talking about two different databases so not sure I can apply your idea. Additionally consider use case where you read data from non-DB data source into KNIME, apply transformation on it and then write into DB. Usually you don’t need this data later on in KNIME.
You are right, I thought you were dealing with tables in the same database.
So, if you have no option than having the input data in AP, then ouput data port of the node is an exact reference of the input table + optional status column. So, if you uncheck the status column then nothing new is calculated. It actually doesn’t make any difference if you had the output data port of the insert node disabled. The “data.xml” under DB Insert only adds this: <entry key="table_reference_ID" type="xint" value="<table ID>"/>
It’s not reading data from DB to AP, it’s just a representation of an existing table.
checked it and this makes a difference. Maybe worth adding to node description that adding two additional columns can impact performance or something similar. Thank you!