File Reader node: bug/error while reading in a tab separated file (KNIME v.4.3.3)

Dear KNIME community,

I am trying to read a .tsv (tab-separated value) file with File Reader node (KNIME v.4.3.3). The data is quite a big one.
While reading in I get the following error:
Execute failed: For input string: “Down, Left” In line 6308 (Row6306) at column #10 (‘Event value’).

The configurations I tried in File Reader node are:
Column delimiter: tab
Quoted strings can extend over multiple lines: check
Ignore extra delimiters at end of rows: check
Allow short lines: check

None of the above configurations has helped.

I wonder whether this has smth to do with the File Reader node bug or not.

The below is the screenshot of how the quick scan of the file looks like:

Has anyone had such a bug? If so, is there any way to fix this?

Thank you!
Odko

Hi @Odko . I suspect that the issue might be with the data. Clearly, the node is able to read the file, and is able to read the data if formatted properly.

Can you try to open the file from nodepad and check the data around Row 6306 - 6308? The most common issue with these files is that there could be some additional \r\n, which is breaking the expected structure of the file.

1 Like

Hi @bruno29a, thanks for the reply. I used to read such files before with earlier versions of KNIME. Hence the thought about a possible bug.

Below I paste the screenshot of the row (open in notepad):

It looks like the comma “,” between “Down” and “Left” is causing the issue. Is there any way I could avoid this issue while reading in the data? Actually, these two values should be read in as a single-cell value.

Thank you again!
Odko

Does the CSV reader do the job? You might need to adjust the number of rows used to guess the spec though.

4 Likes

You could check out this collection and see if one of these ideas does help. Most likely you will have to experiment with the settings in the (new) CSV reader about incomplete lines or you could try and employ the R package Readr which tends to solve most of my CSV problem (examples in the links):

In other news :slight_smile:

https://towardsdatascience.com/stop-saving-your-data-frame-in-csv-format-7823d3873ba2

KNIME supports Parquet and ORC and H2 and SQLite an several other formats …

2 Likes

Hi, @Mark_Ortmann, thanks for your suggestion. Indeed it did work! :nerd_face:

Thank you very much!

Cheers,
Odko

2 Likes

Hi @mlauber71, thank you for your reply. I was thinking of reading the data using R. As R would take quite a long time to read big data, I wanted to go with KNIME. :smiley:

I just tested the suggestion by @Mark_Ortmann above, and it did work.

I will also check the post you shared! Thank you again!

Cheers,
Odko

3 Likes

Hi @Odko , yes most of the time when this happens it’s down to data, and always check it via notepad or something.

I am glad you resolved it via @Mark_Ortmann 's suggestion

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.