Reading from multiple files and adding a path column to each row

Hello everyone. I have several chain problems in my workflow, and I have not been able to solve it myself.

My ideal workflow consists of a loop from a folder that reads csv files and appends a column with the file path to each row from each file.

This would have worked perfectly if it wasn’t for the fact that some files are not being read correctly for reasons unknown, and show weird characters under one of my columns as seen below, and this breaks my loop:

I have read in this forum that this problem does not happen with the File Reader node, and this seems to be true. When I have tried to use the File Reader node however, there does not seem to be a PATH or URL variable to iterate over. I have read that we then need to use a Path to String (Variable) node in order to create a variable after the Loop Start. However, when I try the following:

… the preview in the configuration window seems to be working well, but I get “Errors loading flow variables into node : Coding issue: Cannot create URL of data file from [xxxxxx.csv] in filereader config” and it does not allow me to continue.

I’ve been scouring the forum for days, but now I’m at a loss. Any help would be greatly appreciated!

I have solved the CSV Reader issue with this thread CSV Reader Strange Header - #8 by Vincenzo It seems it should be read in UTC-8 encoding. Now, my workflow functions as expected.

However, while I don’t need to use the File Reader anymore, maybe solving the other problem for future reference could be useful!

4 Likes

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.