Knime processing power blocking at every single step after finished flow

@Gonzo I have no experience with Windows 11 as of now.

These things come to my mind when you mention a large ungroup (or any other complex operation for that matter):

  • the Cache node. Sometimes I have experienced that before complex operations after a long data manipulation stream KNIME ‘likes’ a Cache node in order to bring all the changes back to one place
  • then you could see if forcing the node to write something to disk can help, also if you try different underlying encodings in the knime.ini (-Dknime.compress.io=[SNAPPY|GZIP|NONE]) of the workflows, maybe try GZIP instead of SNAPPY (this might come at a cost in speed and will be the default for every workflow), Also see this entry.
  • try the columnar table backend. KNIME has not yet moved it to a standard but you can activate it for individual workflows
  • and then you could think about splitting the ungroup in chunks (If you data does allow that)

A comibination of these along with some more RAM and garbage collection (1 | 2 ) at the start maybe might bring you across the finish line :slight_smile: