CPUUtilization high and jobs failed - AWS - knime server 4.9

Hi Navin,

As long as the job has not been terminated, KNIME may still hold on to some of the tables in memory. This is intended behavior and not necessarily a bad thing since, for instance, you might want to have a look at some intermediate or final results of the workflow using a table view, in which case it is beneficial to have the data in memory still.

KNIME 4.0 introduced a new table caching strategy that attempts to keep some least recently accessed tables in memory. While this strategy will make your average workflow execution much faster, it will use available memory more liberally and has been observed to severely increase the load of garbage collection on some systems / for some workflows. These changes could be responsible for the increased CPU utilization you are observing. For more details, see this forum post.

If you are having troubles as a result of this change, rest assured to know that we are working on taking load off the garbage collector. If you need a quick fix, you can switch to the less memory-consuming table caching strategy for KNIME 3.7 and earlier by putting the line

-Dknime.table.cache=SMALL

into your knime.ini.

Best,

Marc

2 Likes