Knime 4.7 shuts down abruptly while running a workflow.

[nba]

Hello,

It’s been a few days since I experienced abrupt shutdowns on a workflow that was running without problems previously. The shutdown happens either within a recursive or a group loop.

I tried several solutions to solve that problem, but apparently with no success: I tweaked the.ini file. I cleaned the temp file after several shutdowns. I have re-installed Knime. As the error started around the creation and usage of Linux virtual machines on my computer, I have shut down all VMs, though some VMware services are still running in the back. I updated the JRE and added garbage collector nodes at all critical points in my workflow.

The machine still has the same 32 GB of RAM, with 22 GB allocated to Knime, which is why I find it strange to get a JRE memory error (“There is insufficient memory for the Java Runtime Environment to continue.” from the log)

I attach the log with the error I get as well as my computer configuration. I also paste below my .ini in case it is of any help.

Any help would be highly appreciated.

NB

Edit 1: Since the last shutdown, I cannot restart Knime anymore and got the error message: “Failed to create the java virtual machine”

-startup
plugins/org.eclipse.equinox.launcher_1.6.400.v20210924-0641.jar
–launcher.library
plugins/org.eclipse.equinox.launcher.win32.win32.x86_64_1.2.500.v20220509-0833
–launcher.defaultAction
openFile
-vm
plugins/org.knime.binary.jre.win32.x86_64_17.0.5.20221116/jre/bin/server/jvm.dll
-vmargs
-Djava.security.properties=plugins/org.knime.binary.jre.win32.x86_64_17.0.5.20221116/security.properties
-server
-Dsun.java2d.d3d=false
-Dosgi.classloader.lock=classname
-XX:+UnlockDiagnosticVMOptions
-Dsun.net.client.defaultReadTimeout=0
-XX:CompileCommand=exclude,javax/swing/text/GlyphView,getBreakSpot
-Dknime.xml.disable_external_entities=true
–add-opens=java.base/java.lang=ALL-UNNAMED
–add-opens=java.base/java.lang.invoke=ALL-UNNAMED
–add-opens=java.base/java.net=ALL-UNNAMED
–add-opens=java.base/java.nio=ALL-UNNAMED
–add-opens=java.base/java.nio.channels=ALL-UNNAMED
–add-opens=java.base/java.util=ALL-UNNAMED
–add-opens=java.base/sun.nio.ch=ALL-UNNAMED
–add-opens=java.base/sun.nio=ALL-UNNAMED
–add-opens=java.desktop/javax.swing.plaf.basic=ALL-UNNAMED
–add-opens=java.base/sun.net.www.protocol.http=ALL-UNNAMED
-Xmx22347m
-Xms8g
-Dorg.eclipse.swt.browser.IEVersion=11001
-Dsun.awt.noerasebackground=true
-Dequinox.statechange.timeout=30000
-Darrow.enable_unsafe_memory_access=true
-Darrow.memory.debug.allocator=false
-Darrow.enable_null_check_for_get=false
–add-opens=java.security.jgss/sun.security.jgss.krb5=ALL-UNNAMED
–add-exports=java.security.jgss/sun.security.jgss=ALL-UNNAMED
–add-exports=java.security.jgss/sun.security.jgss.spi=ALL-UNNAMED
–add-exports=java.security.jgss/sun.security.krb5.internal=ALL-UNNAMED
–add-exports=java.security.jgss/sun.security.krb5=ALL-UNNAMED
-XX:HeapBaseMinAddress

hs_err_pid78664.log (216.6 KB)

[takbb]

Hi @nba , do you have any other processes running on your machine that may be using a lot of memory?

[Edit: Having just re-read your post, I would say it highly likely that your system is allocating memory to the VMs and so it cannot fulfil java’s request for the full 22GB.]

(Sorry I don’t have a solution to your current “not being able to start KNIME problem”. The following assumes you manage to get KNIME running again)

I too have experienced sudden shutdown of KNIME through it running out of memory, and whilst I do not know for certain, I believe that the cause in my case it that I had allocated memory to KNIME through the knime.ini file that was greater than the amount that my system was actually able to allocate at the time, because a process outside of KNIME was using up the memory.

The following is my theory on this, so please read it in that context, but it is based on my own observations of this problem…

I believe that allocating, say 22GB of memory, to KNIME means that KNIME will attempt to use up to the 22GB and if it needs more (e.g. 23GB ) it will display an error message but should not exit. So far so good!

It does not mean however, that the system will actually be able to allocate it the 22GB that has been stated in the knime.ini file. I don’t think the 22GB actually gets reserved anywhere.

Now, if KNIME (more precisely java) attempts to use all of the stated 22GB… BUT only 21.5GB is available at that point from the system, I believe that in this case you will see catastrophic shutdown of KNIME as you have experienced.

Do you have the heap monitor showing in your KNIME 4.7.7 UI? (on the status bar)
e.g.

To test this hypothesis, I would suggest turning that on, if not already there, and actually significantly reducing the amount of memory allocated to KNIME, and see if when you run the workflow it then terminates gracefully or still terminates KNIME, when it runs out of memory.

In my case, I had a very large file deletion operation going on in Windows Explorer involving (literally) millions of small files external to KNIME. Windows Explorer sucked up far more memory than I was expecting meaning that KNIME was not allocated all of the memory I said it could have, and so the VM went pop, rather than halting with an error.

I’ve just looked at the log, and sure enough it said the system was unable to allocate more than about 19GB. So I think it fits with my above thoughts. Java thinks it can grab 22GB because that’s what it has been told, so when the system says it can’t have more than 19GB it crashes.

[nba]

Hi @takbb,

Thanks once again for your help.

As Knime was not starting up anymore, I re-installed using 4.7.6 rather than 4.7.7. For now, everything is running fine.

Intuitively, I kind of agree with your theory. However, I struggle to identify what other process could be responsible for sucking up more than the 10 GB of RAM available on top of the 22 GB allocated to Knime. I carefully reviewed all processes running, and it seems rare that the combined usage of all other processes (Knime excluded) exceeds 5 GB of RAM.

I’ll be paying attention in the coming days and will let you know if I can validate or not your theory.

NBA