Java Memory & Thread issue

While executing 40 knime workflow getting below error . (From command line we are executing workflow )
java.lang.OutOfMemoryError: unable to create new native thread

Can anyone suggest on heap size or anything we have to set ?
Configuration -
Available RAM is 256GB

#java -XshowSettings:vm -version
VM settings:
Max. Heap Size (Estimated): 26.67G
Ergonomics Machine Class: server
Using VM: Java HotSpot™ 64-Bit Server VM

#free -g
total used free shared buff/cache available
Mem: 251 55 120 0 75 195
Swap: 99 0 99
in GB

Hi @amar.khavare,

how much memory you have allocated to KNIME in the knime.ini?
->in the knime executor / knime server configuration

How can I increase the Java Heap Space for KNIME?

In the KNIME installation directory there is a file called knime.ini (under Linux it might be .knime.ini; for MacOS: right click on KNIME.app, select “Show package contents”, go to “/Contents/Eclipse/” and you should find a Knime.ini). Open the file,

Find the entry -Xmx1024m and change it to -Xmx4g or higher (for example).

(Re)start KNIME.

1 Like

If we increase -Xmx4g then each workflow will be assigned to 4GB so heap memory issue will come already try this …

You could try and explore this collection about KNIME and performance and see if anything would apply to you

1 Like

Hi @amar.khavare,

just so i understand you correctly - you still get the heap space issue if you increase the space to
-Xmx 200g
?
The 4g is just the example from the faq :slight_smile:
This is not on a workflow lvl but on the executor level as far as i know.
So if you only have one executor you should be able to use a high heap space limit

Yes i tried given multiple sizes like 20GB n all even 200 GB .
As per top command output each workflow will consume same memory like 20GB each workflow .
In our scenario there 40 workflows so 40 executor will be there .

Hi @amar.khavare,

thanks for the confirmation :slight_smile:

So you have a knime server large with load distribution enabled, right?

We dont have knime sever … we are using only desktop version and executing workflow from command line

Hi @amar.khavare,

okay that explains why you need n times the memory… as each batch execution creates a new instance.
But i think then your best choice would be @mlauber71 performance enhancments or spreading out the executions so you have more memory availible for each instance…

1 Like

Thanks @AnotherFraudUser will try with @mlauber71 performance enhancement.

2 Likes

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.