Incremental Data Load

Hi guys

I’m new here and have a question.

I’m creating a workflow where I am executing SQL statements live on a database.
The results are sometimes quite huge (e.g. 85 million rows) in the initial query.
Because of performance reasons I don’t want to handle so much data every time I run it. I would like to implement something that only uses the newest data and adds it to the existing one.

Do you guys have any ideas?

regards and thanks in advance

Hello Kurobasa,
please have a look at this post.
Bye
Tobias

3 Likes

Do you have a sorted field that you can use as boundary?

Hi Tobias
Thank you for the quick response.
That post helps.
Kind regards
Ilija

1 Like