"Lenient" mode for File Reader

KNIME Analytics Platform
1

Hi there,

I'm currently working with some big CSV files, where a very small fraction of lines contains errors (e.g. string instead of number, which is caused by non-escaped column delimiters/missing quotes). I do not care about those ill-formed lines and would just like to skip them, unfortunately the line reader stops at such lines (e.g. "Execute failed: Wrong data format. Got '#0' for an integer. In line 20 (Row18) at column #1 ('LOC').")

Is there a setting, which I might have missed, to make the File Reader "forgiving" such errors?

Best,
Philipp

2

The CSV reader is the most unforgiving and the file reader somewhat forgiving. However, when faced with similar problems on big GB csv/text files, the Line reader node followed by some cell spliter nodes etc work just fine for me. I then export the results as tables for future use and read them using the table reader the next time round.

3