I'm starting my first project with KNIME and I've found a step that I can't solve.
Everyday a csv file containing the energy values from a number of devices has to be read and the new rows appended to a database.
Example with 2 devices:
Date Device 1 Device 2
2013.08.01 70.2 69.7
2013.08.02 75.0 81.9
I've used the unpivot node to store the energy from each device in a different row.
Example after unpivoting the file:
Date Device Energy
2013.08.01 Device 1 70.2
2013.08.01 Device 2 69.7
2013.08.02 Device 1 75.0
2013.08.02 Device 2 81.9
The csv file contains the data for the current month being updated every day, and so when the file is read the next day, each row in the file has to be compared with the rows already stored in the database and only appended those with different date or device.
Can anybody give me a hint about how to do it?
Just done that myself. Basic steps as follows:
Read in the previous version of the table.
Use a reference row filter to exclude any existing records of your workflow output from that previous version's parse
Concat the filtered (previous) records and the new result
Overwrite the entire previous version with the new, concatenated table
Other implementations may be more efficient, but for me the "overwrite" option works fine.
P.S.: Paco OBS/ESS? :-)
Thanks for your help!
P.S.: Paco (Yes) - OBS/ESS (No) ;-)
NP. Just wondering about a Paco ex-colleague of mine. :)
the answer to this question was done 3 years ago. Is there a more efficient way to concatenate avoiding reimporting in knime the table, concatenate the new rows, delete the old table from the database and export the new one in the database? Is there a way to write in the database only the new rows appending them to an already existing table?
Thanks in advance
EDIT. I found the solution. There is a function in the database writer. Thanks