Concatenate multiple files

[dz2709]

Hi!

I have more than 15 files which I would like to merge into one file. My way seems very complex to me.
Is it possible to make this much easier with a Loop?

Thank you for you help!

Best regards

[HansS]

Hi @dz2709

With a workflow likes this.

You can read multiple files at once with a loop structure. I don’t know what is happening in the column filter and column rename, so there may be a (little?) challenge to fix.

gr.
Hans

[ipazin]

@dz2709

Is it possible to make this much easier with a Loop?

Yes!

Br,
Ivan

[chimdee]

I have been playing with these nodes trying to do something similar to dz2709. I have 28 files in a folder that I would like to concatenate, all the folders have same structure with different data. The output in my end loop node is a list of all the records in the excel reader node (the one I’ve right after the start loop node) repeated 28 times. What am I doing wrong?

The additional nodes after the excel reader are cleaning up the data in the excel files and appending the file name to each record (so I know which file ech record came from)

[ScottF]

It’s hard to say without looking at the actual workflow, but that sounds like you’re not passing the flow variable from the loop start node to the Excel Reader node correctly. Not only do you need to connect the two nodes using the red flow variable connection, but you also need to assign the URL for each file in the Flow Variable Tab of the Excel Reader node.

[chimdee]

Thanks @ScottF

I have not adjusted the flow variables in either node. Apologies, I’m a Knime nub, what do I need to assign the URL for each file?

[ScottF]

Take a look at this example workflow from the the Hub. In particular, on the second blue example, check out the Flow Variables tab on the Excel Reader node:

If you need a primer on how flow variables work, this short video should be helpful:

- YouTube