File Reader / CSV Reader

Hello,

I have been trying out the program and it is great - thank you!

I am not sure this is the right place but I wanted to point out something about the File Reader / CSV Reader nodes:

My CSV file does have quotes around each value and in some cases the value can begin with a space (it is rare but it happens - specifically to my case it was in a filename column and some file started with a space however it was important to have the space otherwise those file won’t be recognized or found).

The aforementioned nodes, are forcing a white spaces trim under the “Advanced Settings” tab in the “Quote options” section.

I personally don’t think a change/cleaning of imported data should be made in the same node but if so, I think there should be a checkbox where the first option of “Remove quotes and trim white spaces” is.
Something like:
:radio_button: Remove quotes :ballot_box_with_check: trim white spaces

The way I go around it is otherwise cumbersome:
Import with quotes → use String Manipulator with regex to capture what’s between the quotes on all columns regexReplace($$CURRENTCOLUMN$$, "^\"(.*)\"$", "$1")

I know regex is “expensive” but I have to do all of this because one column might have spaces at the beginning.

Just to point out that I am aware about the Complex Format file reader which in a way can solve it.
However, having that option as is in the CSV/File Reader nodes might cause someone to overlook and get wrong data in case you are dealing with millions of rows.

Thank you for reading and I hope that maybe this can be taken into consideration in the future or at least let other people know of what I see as an issue.

Regards,
Ethan

1 Like