Multiple files .csv reader (Files in folder) problem

[Dalmatino16]

Hello, my name is Dario, and I am a new user of Knime. So far I am thrilled at what it can do and that something like this exists for free.

I have a problem though so I am hoping to find a help here.
I have about 100 csv files. Each file contains data about timeseries, date and time and value of gauged data at the particular time. Here is how it looks like.

I wanted to use Knime to join all this data in the same table, where first column would be date and time and then columns with gauged data from all 100 gauging stations would follow. One colum per gauging station. I browsed the themes here and have found some workflows which use loops, like this for instance:

However, by reading further it seems that the new .csv reader supports “Files in folder” option so loops are not needed for this anymore. So I used that and what I got is this:

Problem with it is that the node has read all the files indeed ( I tried it with 6 csvs only), BUT the values from different gauging stations were stacked one under another - all in one column, while I need each to be in separate column like this:

Can someone help me please? Is there a way to get the wanted result right away from the reader or some extra node is needed? Best wishes, Dario

[mlauber71]

@Dalmatino16 welcome to the KNIME forum. You could take a look at this example:

Loop thru same Excel sheet names from various sources and collect them back –...

You can get information about the name of your file and include that as a constant value in your workflow to add to your table.

Maybe you could provide us with a small example that would represent your challenge.

[Daniel_Weikert]

@Dalmatino16
You need to replace your loop end node with a loop end (column append). That stacks the data column instead of row-wise
br

[Dalmatino16]

Hi mlauber and thanks for trying to help. Attached find example of inputs.

Best wishes
H_daily_3018.xls (234.7 KB)
H_daily_3352.xls (234.3 KB)
H_daily_4061.xls (229.6 KB)

[Dalmatino16]

Thanks, will try that too

[Dalmatino16]

Workeld like a charm Thanks again.

[Dalmatino16]

Actually I just have noticed that I still have a problem. A different one. I ended up with columns containing the same data

The workflow should have gone through different files, but it seemed to went over and over again the same file. Here is the workflow. Any suggestions?

[Thyme]

Have you set the File Reader to use a Path variable instead of a static filepath (see screenshot)?

The button is supposed to look like it’s pressed, with a notification in the bottom mentioning that the path is controlled by a Flow Variable.

[Daniel_Weikert]

@Dalmatino16
I think @Thyme is on the right track. You can check that.
br

[Dalmatino16]

Hi, thanks for the hint. You got it right !

[Dalmatino16]

Hm. It worked while I had 3 files in folder. But now that I have 25 files in the folder the option is grayed out and I can`t tick it. Any idea why?

[Dalmatino16]

I guess it doesn`t have anything to do with numbers. I tried again with only three files and the option is grayed out again. First time I was able to set it up. Whats wrong?

[Thyme]

Like the dropdown says: You don’t have any suitable variable available. The File Reader needs a Flow Variable of type “Path”.

How does the output of that List Files/Folders node look like?

[Dalmatino16]

Here is how it looks like for three files in the folder

[Dalmatino16]

I figured out whats wrong. I didn`t run Node 15 before I was trying to set up File reader config. Thanks so much.

[Thyme]

Glad it works now. Don’t forget to select a post as solution so others know what fixed your issue

[Dalmatino16]

I did, but too bad I can`t select several posts, because several of them resolved the issue combined