Reading multiple csv files

[beginnerknime]

Hi

i have a simple workflow that reads from 1 csv file takes the data i need and writes it to an output file.
How can i feed my workflow with a list of 100s of csv’s so i can build my output file?

Thanks

[johannes.schweig]

Hi @beginnerknime,

Your List Files node is a good starting point. There is an example on that on the example server: 06_Control_Structures/04_Loops/02_Example_for_Reading_a_List_of_Files

Reading multiple files using loops – wiswedel

Let me know if you can’t get it to work.
Cheers,
Johannes

[beginnerknime]

It worked.
Thanks

[beginnerknime]

csvfile test.knwf (25.9 KB)

Can you have a look at this workflow.
I am trying to read from multiple csv files into 1 table -> select columns i want to filter -> get the max value from each column an generate a table to run stats from.

I had this working before i had to do a reinstall, but cant get it to work now as it just keeps getting the data from 1 csv file.

[johannes.schweig]

Do you get any error messages from the nodes?
How does your data look like after the loop end? For me it is one row for each datafile.

[beginnerknime]

WARN File Reader 0:10 Errors loading flow variables into node : Coding issue: Cannot create URL of data file from ‘’ in filereader config

I get this error

[mlauber71]

It is difficult to see what is going on. You could try to modify this example and import the CSV files via Readr package which is more tolerant/smart with regards to messy CSV files. Also you should make sure you browse the right folder and there are no strange files.

Error with File Reader node while running in a loop

I had a similar problem where there were some ‘dirty’ CSV files out of a large number of files and they were too large to just ‘edit’ them in Notepad++ (and that also was not a very nice process). I built a workflow using the R package readr, which seems to me more tolerant towards problematic CSV files and gives you some control how to handle mismatching quotes and so on. You might have to toy around with the settings of the read_delim function. [46] kn_example_import_csv_r_readr.knwf (52.…

You column aggregator node suggests there were quite some different columns present before.

And maybe you check if this is the right folder:
C:\Users\xxxxxx\Desktop\New folder (2)

Of course you are free to name your folders like you want but maybe you have a path that is more significant and try to loose too much special characters , blanks and brackets - could be the URLs do not like that, even if they should be able to handle it.