Concatenate table as SQL UNION

Hi,
I have two CSV files generated by two monitoring programs. The structures of both files are identical, except that one program write an header row with field names, the other program dumps the data directly.

I cannot find a easy way to merge the two tables in only a longer one because Concatenate node produces a table with both named and anonymous column.

E.g.

File 1:
FieldA,FieldB
1a,1b
2a,2b

File 2:
1c,1d
2c,2d

I would like:

FieldA,FieldB
1a,1b
2a,2b
1c,1d
2c,2d

Instead, Concatenate node produces:

FieldA,FieldB,col0,col1
1a,1b,?,?
2a,2b,?,?
?,?,1c,1d
?,?,2c,2d

I understand the logic behind it, but I am wondering if there is a way to merge the two tables without work around (e.g. R scripts, string transformations, etc…)?

With the Rename node you can change the names “col0” and “col1” to “FieldA” and “FieldB” before using the Concatenate node. This will solve your problem.

Thank you, but this is exactly a type of workaround that I would avoid.
The files have a lot of columns and moreover, I could not know in advance their number nor their names.

I guess you used two File Reader nodes to get the data into KNIME, aren’t you? Here is my list of possible workarounds:
(1) Ignore the column header, use the option inside the File Reader dialog. All columns will have the default name as “ColX” as generated by the File Reader (The original column names get lost!). Filter out the first line from this table. Concatenate…
(2) For both branches, use the Column Combiner, Concatenate, and then Column Splitter node. (The column names are changed into “comb_Attr[X]”!)
(3) Use two CSV Writer, one generates a new data file with column header, the other one appends only the date to the same file without column header information. (Both writers must be executed in the right order!) Read in the concatenated/appended file.
(4) Even better as in (2), instead of using the Combiner and Splitter use the Create and Split Collection Column nodes. The Splitter has an option to use the column information (name and type) from the input table. Would be my choice.
(5) Wait till the next release where we will have a meta-node to iterate each column which can then be renamed during each loop.
Good luck.

Just for fun, I developed a node that paste together two tables.
You can download source and plugin from http://slipguru.disi.unige.it/Resources/knime.php

That link appears to be broken, but I found the node on his site at this link in case anyone else needs it:

http://slipguru.disi.unige.it/index.php?option=com_content&view=article&id=29:knimeplugin&catid=10:software&Itemid=29

Now that I've tried the node, it didn't work for me. I wanted to union tables with 227 and 242 rows, none of which should have been filtered out as duplicates, but the 'merged table' only had 242 rows.

Hi,

This can be done with KNIME's standard nodes.

If the two tables have identical column names, simply use the Concatenate node in Data Manipulation/Row/Transform. Use the default settings in the node, of Union, and appendixing suffix to duplicate row names.

If the two tables have non identical column names, simply use the "Extract Column Header" node in Data Manipulation/Row/Other.  Simply use the output from the second port of this node into a Concatenate node. Thats it.

Simon.

Thanks, Simon. That works great. I guess the term 'Concatenate' threw me for a loop...thought it would do something different, but nice to be able to use it to simply Union.