Prevent duplicate data entry in SQL through KNIME

[alex1368]

I have a lot of data and information that is added daily, which is too much and cannot be stored in Excel.
I want to enter that information into the SQL database and use it later
I want to prevent duplicate data entry (based on two columns) during data entry.
How can I do this through KNIME?
I use the following nodes to enter information

[mlauber71]

@alex1368 typically a database would prevent duplicate entries thru PRIMARY keys. You might want to check your database and configure it that way. You could also do that with a local H2 database for example. Then you might have to decide what to do if you encounter a duplicate key

You can also check in advance if your database already has the keys you want to insert beforehand and then act accordingly.

These examples might be worth exploring. But maybe you give us some more details about what you ant to do and maybe even some dummy/sample data to illustrate your point.

How to update database with scraped data

@RIchardC I put together a (hopefully) complete example using H2 and the Northwind database with customerid. Here we first create a database with a customerid as a primary key (randomly selected from the central DB). Then every 10 seconds another random batch gets drawn. Existing customerid will be updated. Then the workflow determines which customerid are new and would insert them. If a row is inserted initially a timestamp first_inserted will be stored. Then with every update another timestam…

https://hub.knime.com/search?type=Workflow&tag=update,sql&sort=maxKudos

How can I accomplish on KNIME "INSERT ... ON DUPLICATE KEY UPDATE Statement"?

@bilgee I would like to point you to a KNIME-only solution to your task since that might be what you initially asked [image] Also I created an example using H2 database and Stud_id as PRIMARY_KEY and load everything into a new database [image]

Example of H2 database - handling of database structure and use of Primary...