Slow performance of my database

[chlovi]

Hi everybody,

I’ve got some troubles with the analysis of my database of 34 million observations.
It takes many time to calculate a node (for example, a simple row filter node takes around one hour). How can I improve the performance of KNIME ?

Here is my computer features :

Thank you in advance.

Chloe

[izaychik63]

Take a look on discussion here

Reading from databse

Hi there @pruthvi1286, print screen @izaychik63 posted is from new database nodes available in KNIME 4.0 version. As for legacy database nodes you are using there is no fetch size option that can be configured inside node. Instead fetch size can be configured in knime.ini configuration file. Check this topic: Poor performance in Database Reader Node Additionally network connection can be bottleneck so check that as well. What database are you using? I suggest updating to newest KNIME version…

[beginner]

What exactly is slow? The reading of the data from the database or the processing in KNIME? As far as I understand it’s the actual processing in KNIME?

What you can do is increase the RAM you give to knime in knime.ini. See this. You could try to up it to 5-6 GB. But for that you need to close all other things on your laptop including the browser.

Also depends what your row filter is actually filtering. A regex filter will probably be a bit more intense than a simple numeric range filter.

But regardless of that, 8 GB RAM simply is a little low if you are doing “data science stuff” with 34 million rows. On top of that your processor is from a 15W laptop, eg small form factor laptop
with a very low base frequency of 1.8ghz. You can’t expect wonders from this setup. And what you aren’t showing at all is your storage, If this laptop has a HDD (and not ssd) combined with the small amount of RAM the poor performance makes a lot of sense.

[chlovi]

Thank you for your help.