Xmx parameter doesn't prevent JVM using RAM beyond it's limit.

KNIME Analytics Platform
1

In reference to this topic I have exactely the same problem under linux but not on Windows : Memory problems causing server to shut down KNIME (it's driving me crazy) - #27 by christian.birkhold

Problem : tests are done on a windows laptop (32G ram it works Xmx 20G) but production environment are under Linux (48G RAM Xmx 40 or 20 or 30 whatever Memory grows to 48 freezing shutdown under 5.3 => 5.8 Knime version). So I ask the question again, why Xmx parameter doesn’t work on linux ?

image
image1920×1077 97.2 KB

1 Like
2

At least is there an old linux reliable version ?

3

Dear @Fabien_Couprie,

Thank you for reporting. I will share this internally and come back to you soon.

1 Like
4

Using the Xmx command line parameter only sets the maximum size of the Java heap. However, there are many more memory pools that Java uses and there is a significant amount of internal memory needed for the JVM (e.g. garbage collection as well). Therefore the memory footprint of the Java process can easily exceed the configured heap size by 30-50%.

5

Well, how can it be possible that the process can run without failure on windows with 32 GB RAM and fully use a 48 GB RAM on linux forcing the server to shut down. What parameter should I use to prevent this issue ?

6

Well, this depends on many things:

  • Is the configuration really 100% identical? Check whether there any other memory settings in the knime.ini.
  • Are you processing exactly the same data with the same workflow?
  • Are your columnar storage settings in both installations identical?

Usually using 60% of the available host memory for Java heap is working fine. But of course there can be exceptions.