I am trying to write data into a table on sql server 2014 using a database writer node and it takes 5 hours to write ~135k rows with 130 columns. Is there a better/faster way/node to insert into tables. The data is a Knime output table from an R snippet node. Thanks in Advance!
I have same issue,
At the moment, we don't have any other option unless they can develop new node.
I see that so many users are complained but nothing improved yet
in case I am using Alteryx for time being and it is very fast.
I hope they can develop similar like that
The problem is that the Database writer node has Batch Size set to 1 by default. Set it to something higher like 100 or 1000 and you will get an according speed up.
But yeah, database stuff could improve especially transaction handling.
Awesome. That makes is so much quicker. I insert about ~100k rows at a time and not more so this helps a lot.
Hello – When I attempted to use this solution, of increasing the batch size, what happened to me is that the writer started skipping rows with this increment, so for example of I have 500000 rows of data, and a batch size of 100000, I would end up with a table with only 5 rows. Any idea if I need to set something up differently to make it work? My database is on Microsoft Azure in case that helps.
A post was split to a new topic: DB Update node too slow