Adding new rows to a database

KNIME Analytics Platform
1

Hello,

I'm starting my first project with KNIME and I've found a step that I can't solve.

Everyday a csv file containing the energy values from a number of devices has to be read and the new rows appended to a database.

Example with 2 devices:

 

 

Date       Device 1 Device 2

2013.08.01     70.2     69.7


2013.08.02     75.0     81.9

 

I've used the unpivot node to store the energy from each device in a different row.

Example after unpivoting the file:

 

 

Date       Device   Energy

2013.08.01 Device 1   70.2


2013.08.01 Device 2   69.7


2013.08.02 Device 1   75.0


2013.08.02 Device 2   81.9

 

 

The csv file contains the data for the current month being updated every day, and so when the file is read the next day, each row in the file has to be compared with the rows already stored in the database and only appended those with different date or device.

Can anybody give me a hint about how to do it?

Best regards

 

Francisco

2

Just done that myself. Basic steps as follows:

  1. Read in the previous version of the table.
  2. Use a reference row filter to exclude any existing records of your workflow output from that previous version's parse
  3. Concat the filtered (previous) records and the new result
  4. Overwrite the entire previous version with the new, concatenated table

Other implementations may be more efficient, but for me the "overwrite" option works fine.

 

Cheers

E

 

P.S.: Paco OBS/ESS? :-)

3

Thanks for your help!

Francisco

P.S.: Paco (Yes) - OBS/ESS (No) ;-)

4

NP. Just wondering about a Paco ex-colleague of mine. :)

 

E

5

Hi,

the answer to this question was done 3 years ago. Is there a more efficient way to concatenate avoiding reimporting in knime the table, concatenate the new rows, delete the old table from the database and export the new one in the database?   Is there a way to write in the database only the new rows appending them to an already existing table?

Thanks in advance

Regards

 

EDIT. I found the solution. There is a function in the database writer. Thanks