loop files in a folder with join rather than concatenate

I have multiple .csv files which all of them in 1 folder such as table1.csv, table2.csv, table3.csv… this is just an example, more than 3 files in the the folder. They have the same ‘id’ but all of them have different column name.

I try to write a loop use the following node: list files/folder → table row to variable loop start → csv reader → loop end, but the loop end say execute failed: input table structure different from reference (first iteration) table. I think it is just using for concatenate the table.

How could I loop files in a folder with join rather than concatenate? (i.e. the desire output table that I want)

Table1 to 3 csv and the desire output is as following:

table1.csv:
image

table2.csv:
image

table3.csv:
image

the output that I want:
image

Thank you!!!

Hello @vansa211,

and welcome to KNIME Community!

You can use Loop End (Column Append) node for your use case.

Br,
Ivan

6 Likes

Thanks! Loop End (Column Append) is useful!
I would like to know how could I only have one column of the ‘id’?
After Loop End (Column Append), it has column with id, id (Iter #1), id (Iter #2)…

Hi @vansa211

Add a RowID node inside your loop, and replace the KNIME Rowid with your id-column.

image
gr, Hans

2 Likes

I try to put the row ID node inside the loop, but there is no column for me to choose:
list files/folder → table row to variable loop start → RowID → csv reader → loop end (Column Append)

image

Hm,

Did you execute the first part of your wf?
list files/folder → table row to variable loop start,
and then configure the RowID node…

yes, of course. I execute list files/folder → table row to variable loop start.
any setting should I set at the 2 nodes?
Seems nothing that let me set…

or could I remove those column after the loop?

FYI, the ‘id’ may not in a continue numeric order, it may be 1,2,3,6,13,15,16, 20…

Hello @vansa211,

you can carry out your loop as you did and use Column Filter after Loop End to leave only one (first) id column and remove all other containing word Iter. This can be done using regular expression in Column Filter node. Here is screenshot of flow together with Column Filter configuration.

For more on regular expression negative look-arounds see here:

Br,
Ivan

1 Like

thanks!! it is work!

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.