Python script node error when executing in the workflow only with 38M rows, but runs fine at a few million.

[mlauber71]

@RVC2023 you can check general performance settings especially your RAM (memory) so that KNIME does have enough.

If you must you can split data you export (or import) from KNIME to Python into several Parquet files and (later) handle them as one big file:

use the new (KNIME 4.6+) Python Script node and bundled Python version to...

Mor on the processing of Big Files

KNIME Snippets (1): Collect and Restore — or how to handle many large files...