Catching nodes that takes too long or never completes

[havarde]

While KNIME is typically quite stable, we regularly have situations were nodes never complete. This could be External Tool Nodes or most typically DB Reader Nodes accessing remote databases (most of the databases we are using are cloud based today and thus “remote”).

Very often the situation may be easily handled by backing up a step and retrying. However, today this must be done manually as we have not found a way to create a kind of timeout error that we could catch in the workflow.

Is there any clever way to do this?

[mwiegand]

Hi @havarde,

What about the timeout advanced settings for polling and timeouts?

If that doesn’t work, your could try the Try Catch Nodes combined with a Generic Loop Start and Variable Condition End? You can force loop until a condition is met. Leveraging the Wait node alongside the aforementioned you can set an artificial timeout which would trigger the Branch Inverter an thus the reiteration.

Try (Data Ports)

Catch Errors (Data Ports)

Generic Loop Start

Variable Condition Loop End

Wait...

Active Branch Inverter

I have a workflow which saves intermittent results, forced re-execution but putting the failing iteration at the end of the stack plus collecting the exceptions thrown. A very rudimentary example could look like this:

The main challenge however I see could come form calling remote workflows but not stopping their execution.

Adding to this, the Vernalis nodes offer some great timing based loops:

Timed Loops

Hope this helps or at least provides you some food for thought.

Cheers
Mike