Controlling memory issues

KNIME Analytics Platform
1

Hi,

there are certain nodes which consume a lot of memory, and can max out the heap space, the clicking of the bin doesn't always release this memory during the operations. Whilst this heap space is maxed out, knime grinds to a near halt, the knime window doesn't redraw or redraws very slowly, and can become quite unresponsive. And yes, I have the heap as large as it can be set on 32bit windows.

now sometimes this happens when you least expect it, and a lot of time and work has been lost as it's either resulted in knime crashing, or being so slow and unresponsive that an end task from windows is needed.

I am no programmer, so I don't know if it's possible, but can Knime be programmed such that it keeps a little bit of this heap space selfishly for itself in the running of the main part of knime, (which the nodes cannot use up), I.e. So that The GUI part and it's other functions like saving and the stop workflow buttons continue working at full capacity even when the nodes have consumed all their allocation of memory.

thanks,

Simon.

2

Unfortunately this is not possible. We have not control whatsoever of how much memory an individual nodes uses. We can (and do) only control the buffering of input and output data.

Usually if memory gets really low at some point in time the node is violently stopped by Java and then free memory is available again. This make take some time, however and KNIME looks like it's frozen because Java is busy trying to free memory. So you don't necessarily need to kill it, you just need to be patient (which is hard, I know...).