Hello @richards99,
could you give a screenshot of the problematic part of your WF (without any sensitive data) maybe if we have a look we could suggest some workaround or other (less memory need) logic for your WF.
In the meantime you can investigate the “usual things” when memory issues happen:
- playing around a little bit with the heap space memory of you KNIME (in the knime.ini setting the xmx value higher) KNIME is rather column sensitive than row sensitive, probably that is why you experience the same issue independently the number of rows but the same column number. In the KNIME Workbench Guide you can find how to set the xmx value and other settings possibilities as well, it worth to have a look.
- try to write data into disc and not keep it in memory
- if it is possible, try to transpose your data (limitation here: this is only useful if your columns are all of the same type and you have fewer rows than columns)
- in this quite old but still valid blog post, and in this and this threads you can find many other tips and tricks, how to handle a memory issue.
Please let us know if it helped you!
Regards,
Dora