Trying to read a .txt file with delimiter from database extract

[Page0727]

I am trying to read a .txt file with @^@ as the delimiter. The most success I get is from using the file reader (complex format) node but once it gets to a particular row it has an error. It seems to relate to a column that contains a lots of data that goes over quite a lines within the text file. It appears between the delimeters but the amount of data seems to confuse the reader. Is there another setting that I need to apply?

[mlauber71]

@Page0727 you import the file as one large column with the Fixed Width File Reader – KNIME Community Hub. Then you replace your separator with a separator like ASCII 164 (¤). Then you export it as a CSV with blank as column separator. And then you import it back into KNIME with CSV reader and the new separator.

CSV Writer adds random rows while creating the file

@pranavgupta you could try to use R to write the CSV file and see if there is a different outcome. CSV files are not ideal when transferring data but I do understand why they are still being used. Sometimes I used even more excotic characters as separators like the “flattened turtle” ¤ ASCII 164. And you definitely should check for some hidden line breaks in the original data.

Also you could try to then allow short lines.

[Page0727]

I came across where the error is and managed to get it to work using python script.
The problem it seems is essentially as you suggested there are some fields that have multiple lines of data with each one of those starting with a \t. Is there anyway to incorporate this in the file reader node?

Thank you for your response and apologies for not getting back to you sooner.