Knime Ram Usage

KNIME Analytics Platform
1

Hi all,

we are working on a workflow for data deduplication.We found that similarity search node which is essential for our flow is taking  a lot of time to process dataset with 1million obs.so we thought increasing ram would increase the knime performance and we started working on 24gb ram.Later we observed that the ram usage of knime is not crossing 8gb whatever the case maybe.is there any cap on ram usage in knime? and why its not using the entire memory when it is provided with complete ram and even i have allocated the entire ram in knime.ini!!!

2

Hi vihar,

If you have already adjusted Xmx in knime.ini, the next step should be to increase the number of cells KNIME keeps in memory before writing to disk. In your case, I suggest trying with 1million, the default is 100k.

You can do this by adding -Dorg.knime.container.cellsinmemory=1000000 after the -vmargs argument in your knime.ini

Hope that helps!

Cheers,

Roland

2 Likes