I am attempting to use the database writer node to write to a MySQL database however, when executing the node, it is extremely slow -- roughly 10 rows/second.
The profile of my data is 15 columns, 255K rows -- primarily varchars of various lengths.
After browsing the net, I have added the following to my INI file.. I would note that the fetch size has DRAMATICALLY improved my performance using the Database Reader node.
with the database.fetchsize you already found the parameter to tweak db-performance within KNIME. Everything else mostly relies on the network connection or the DB itself. This topic has also been discussed in the following thread: https://www.knime.com/forum/knime-users/knime-performance-reading-from-a-database (which I assume you already read).
Hey there, I had the same issue… and was able to solve it really simply, KNIME has a KNIME.ini file, this one is like the paramethers KNIME uses to execute…
The real issue is that JBDC driver is set for 10 Fetch Size. By default, when Oracle JDBC runs a query, it retrieves a result set of 10 rows at a time from the database cursor. This is the default Oracle row fetch size value… so whenever you are reading database you will have a big pain waiting to retrieve all the lines.
The fix is simply, go to the folder where KNIME is installed, look for the file KNIME.ini, open it and then add the following sentences to the bottom, it will override the defauld JBDC fetching, and then you will get the data in literally seconds.
