Memory Issue Knime Version 5.3

KNIME Analytics Platform
1

Hi everyone,

I’ve been experiencing memory problems with KNIME 5.3 since updating. I’m running KNIME on a laptop with 8GB RAM and a 2GB graphics card.

I’m frequently getting “Out of Memory” errors, especially when working with workflows that involve Python nodes. I’ve attached my knime.log file for reference.

I’ve already tried modifying the knime.ini file (see below), but the issue persists. I’ve noticed that KNIME seems to be launching multiple Python processes, which might be contributing to the problem.

Here’s my current knime.ini configuration:

-vm
plugins/org.knime.binary.jre.win32.x86_64_17.0.5.20221116/jre/bin/server/jvm.dll
-startup
plugins/org.eclipse.equinox.launcher_1.6.700.v20240213-1244.jar
–launcher.library
plugins/org.eclipse.equinox.launcher.win32.win32.x86_64_1.2.900.v20240213-1244
–launcher.defaultAction
openFile
-vmargs
-Djava.security.properties=plugins/org.knime.binary.jre.win32.x86_64_17.0.5.20221116/security.properties
-Djdk.httpclient.allowRestrictedHeaders=content-length
-Djdk.httpclient.allowRestrictedHeaders=content-length
-Dorg.apache.cxf.bus.factory=org.knime.cxf.core.fragment.KNIMECXFBusFactory
-Dorg.apache.cxf.bus.factory=org.knime.cxf.core.fragment.KNIMECXFBusFactory
-Djdk.httpclient.allowRestrictedHeaders=content-length
-Dorg.apache.cxf.bus.factory=org.knime.cxf.core.fragment.KNIMECXFBusFactory
-Dorg.apache.cxf.transport.http.forceURLConnection=true
-Djdk.httpclient.allowRestrictedHeaders=content-length
-Darrow.enable_unsafe_memory_access=true
-Darrow.memory.debug.allocator=false
-Darrow.enable_null_check_for_get=false
-Dorg.apache.cxf.bus.factory=org.knime.cxf.core.fragment.KNIMECXFBusFactory
-Dorg.apache.cxf.transport.http.forceURLConnection=true
–add-opens=java.security.jgss/sun.security.jgss.krb5=ALL-UNNAMED
–add-exports=java.security.jgss/sun.security.jgss=ALL-UNNAMED
–add-exports=java.security.jgss/sun.security.jgss.spi=ALL-UNNAMED
–add-exports=java.security.jgss/sun.security.krb5.internal=ALL-UNNAMED
–add-exports=java.security.jgss/sun.security.krb5=ALL-UNNAMED
-Dorg.apache.cxf.bus.factory=org.knime.cxf.core.fragment.KNIMECXFBusFactory
-Dorg.apache.cxf.transport.http.forceURLConnection=true
–add-opens=java.xml/com.sun.org.apache.xerces.internal.parsers=ALL-UNNAMED
–add-opens=java.xml/com.sun.org.apache.xerces.internal.util=ALL-UNNAMED
-server
-Dsun.java2d.d3d=false
-Dosgi.classloader.lock=classname
-XX:+UnlockDiagnosticVMOptions
-Dsun.net.client.defaultReadTimeout=0
-Dknime.xml.disable_external_entities=true
-Dknime.table.cache=SMALL
-Dcomm.disable_dynamic_service=true
-Dp2.trustedAuthorities=https://update.knime.com
-Declipse.e4.inject.javax.warning=false
-Dorg.eclipse.ecf.provider.filetransfer.excludeContributors=org.eclipse.ecf.provider.filetransfer.httpclientjava
–add-opens=java.base/java.lang=ALL-UNNAMED
–add-opens=java.base/java.lang.invoke=ALL-UNNAMED
–add-opens=java.base/java.net=ALL-UNNAMED
–add-opens=java.base/java.nio=ALL-UNNAMED
–add-opens=java.base/java.nio.channels=ALL-UNNAMED
–add-opens=java.base/java.util=ALL-UNNAMED
–add-opens=java.base/sun.nio.ch=ALL-UNNAMED
–add-opens=java.base/sun.nio=ALL-UNNAMED
–add-opens=java.desktop/javax.swing.plaf.basic=ALL-UNNAMED
–add-opens=java.base/sun.net.www.protocol.http=ALL-UNNAMED
–add-opens=java.base/sun.net.www.protocol.https=ALL-UNNAMED
–add-opens=java.base/java.util.regex=ALL-UNNAMED
-Xms1024m
-Xmx4096m
-Dorg.eclipse.swt.browser.IEVersion=11001
-Dsun.awt.noerasebackground=true
-Dequinox.statechange.timeout=30000
-Dorg.knime.container.cellsinmemory=4000

Any suggestions on how to resolve this would be greatly appreciated.

Thanks!

2

4 GB of RAM is not that much these days. You can try to increase that to 5 or maybe even 6 GB if you would not have anything else running.

Also you should check some options to speed up your processes

1 Like
3

I am also seeing this issue on Knime Business hub when working with python.

Execution Context has 18GB of RAM assigned but Python script nodes are failing with this error:

The Python process was killed to prevent the system from running out of memory (it used 4214MB).

4

Thanks for the help . I read the post and made some adjustments, and it improved the performance. I was used to knime in version 3 and 4, and the memory consumption was not as high as version 5. I am doing a RAM upgrade and for now, I can’t open another application while using knime.

1 Like