Best way of merging tables from files

KNIME Analytics Platform
#1

Hi everyone,

currently searching for a handy way to merge tables/files. I'm trying to merge comparable tables (a list of feature with associated categories, two columns each file)

term -> category
(one term) -> category A
(another term) -> category B
[...]

I have about 7 categories and several thousand words, lets make it 20 csv-files for example. In every file every category is present. What I would like to do is merge all 20 files, then group it to remove duplicates and then filter by nominal categories (easy with the nominal value row‑filter-node) and generate one new document for every category.
What I’m doing right now is to use a concatenate node for each and every new document. This works but I guess there are better ways to solve this problem.

What are easier ways of merging tables from several files?
For some files I have to filter some rows before merging so the ideal solution includes the possibility to do some individual “preprocessing” on the tables.

#2

Hi Tim,

you can loop over the files. The List Files node gives you a list of all files in the folder and the Iterate List of Files Meta node shows you how to read them with flowvariables.

But if there is some individual preprocessing... you can try with the Switches in the flow control, but this might get quite complex.

Cheers, Iris

#3

Hi Iris,

thanks for your quick reply. I guessed that it might be hard to do some individual preprocessing, so I guess the easier solution it to harmonize the file structures beforehand.

I assume you mean some looping approach as mentioned e.g. here in the KNIME forum
this page

Gosh, there is quite a number of solutions to the problem...

Thanks for your answer! (and sry for asking a rather redundant question)