I have a KNIME Workflow that reads a few large gzipped CSV files (largest is 700mb zipped, 35 million lines).
The workflow works locally in KNIME AP. The machine running KNIME AP has 16 GB memory, I do not see excessive RAM usage.
I deployed the workflow to KNIME Server, the executor has 20GB Memory. I constantly run into errors in the file parser nodes. The error says
java.io.IOException: Premature EOF but I checked the gz files on the server, they are complete, can be unzipped and show
gzip -t -v.
Maybe I’m missing something obvious. Can this be a memory problem, maybe because the memory/caching policy on KNIME Server is different from KNIME AP?
The full log is attached, any help would be appreciated.
knime.log (36.2 KB)