Hi Erich,
Thanks for the workflow, we have reproduced and identified the problem and created a ticket internally so that we will try to fix it for the next bugfix release of KNIME, so 4.5.2.
What happened in a nutshell is that we were keeping track of some objects to be able to close them properly (=free their memory) before they get garbage collected. However, in your nested loops, the amount of objects that we are keeping track of accumulates to such an extent that we run out of heap space. We’ll have to reduce the amount of objects created and need to improve the tracking of closeable objects.
Unfortunately that means you will have to use the “default” backend for now, but we’ll let you know as soon as we’ve released a fix for this problem in the “columnar” backend!
And regarding KNIME 4.3.1: At that time the Columnar Backend was still in “Labs”. It has changed and improved significantly in the meantime and thus should work much better. Actually, tracking the closeable objects as mentioned above is in place to prevent the memory leak detected by the Arrow Allocator. Unfortunately the amount of references grew too large in your workflow.
Cheers,
Carsten