Pause - Resume nodes and loops over knime or system restarts

As the title says. If you have a long running node or loop it is often needed to cancel and retry at a later point potentially over night to be able to finish execution.

In these cases it can be extremely helpful to be able to resume, even after restating knime or the laptop. As a first step for loops ,maybe later for nodes?

On top of this being able to see temporary results would also be very, very beneficial. You can’t look at any result until the loop is completed. for example parameter optimization loops this could be very helpful.
(yeah you can hack something together to write results out and append but that will just make the loop also run slower due to additional IO)

if it works for loops one could at least but a long running node into a Chunk loop and get Pause - Resume that way.

1 Like

Hi @kienerj,
Thanks a lot. This really is a good suggestion.

Thanks,
Sanket

1 Like

Hi @kienerj, in addition to your suggestion, the idea of Step Loop Execution is possible in the Loop Nodes. This could help if the data is chunked, and the intermediate results could also be monitored.

You can find more details in the KNIME Flow control guide:
https://docs.knime.com/latest/analytics_platform_flow_control_guide/index.html#_loop_commands

Thanks,
Sanket

1 Like

Yeah I’m aware of that but it’s really about Resuming loop execution after knime and system shutdown/restart.