Good day
From time to time, when handling extremely large data sets, I might inadvertently run a workflow which takes an extremely long period of time to read when reaching the DB Reader node.
On occasion I’ve tried to cancel this node when I see the process will take too long but frequently the cancelling of this node takes upwards of 15minutes, especially if I catch it too late and it’s already read a fair few rows. It still takes a large amount of computer resources until it finally cancels the read, sometimes causing KNIME to stop responding periodically.
My question is this, is there a way to force the node to cancel or fail, say perhaps by terminating a process in task manager or something?
Image attached for reference. Cancelling the node in the image took about 20 minutes. I’m working on a server is more than enough resources.
Hi, this is a know issue with knime. Some nodes take a long time and there is no way to cancel them.
The only way I found is to shut down knime, but that sometimes leads to data los…
@Ntrpy what kind of database is this and 1 million lines is a lot and it might also depend on your network speed. In general you might think about doing it in chunks or since you use a group by maybe processing that on the database itself and only load the results.
What you could try is open another branch and close the connection and see if the query then fails
One idea could also be to send some sort of stop signal via SQL executor depending on the database. Would have to see if multiple operations are allowed and can be configured.
