Concatenate historical data

Hi together,

I have the following use case:

I have historical data (lets say from the year 2019) and I want to combine them with the new data from 2020. The structure of both datasets is identical. The historical data will be imported via a table reader node and the current data will be transformed via a KNIME workflow.

My idea is, to use a Concatenate node, but I’m not sure how to configure it correctly to leave the historical data untouched and just add the new data?

Or is there another approach to do this?

Thank you and regards,

Andreas

Hello @Schmali93,

you should use Concatenate node for this. It does not change data. So if the structure of both datasets is identical then it will simply append data coming from second input port to data from first input port.

Br,
Ivan

1 Like

Hi @ipazin,

thank you for your hint regarding the Concatenate node.

Maybe you can also give me a tip regarding the configuration of this node.

I’m not sure which setting fits the best for my use case. At the moment I use this ones:

image

Best regards,

Andreas

Hello @Schmali93,

I would use Append Suffix option otherwise you can loose some rows in case of duplicate RowIDs. Same goes for Union option as if you have intersection option and different data structure you’ll loose columns.

Br,
Ivan

1 Like

Hi @ipazin,

thank you for your feedback again!

I will try your proposal.

Hopefully this will work after the switch from 2020 to 2021 :wink:

Best regards,

Andreas

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.