Pause Loop Execution Conditionally / Programmatically

Dear Knimers,

who hasn’t face the situation where a loop execution fails or you just want to inspect the data let’s say every then iterations?

As of today there is no node available to just halt a loop execution. Thankfully, due to flexibility of Knime but also the many awesome extensions, a solution is only some thoughts away.

Please find hereby a showcase of three solutions how to conditionally / programmatically pause execution.

Utilizing a little bit of Python, in curtesy of ChatGTP as I am not a developer, the first two open a modal drawing attention the the Knimer.

The second even provides some basic inspection capabilities.

The third leverages a simple pause / lock file in conjunction with the wait node.

Please let me know your thoughts and how to improve it?

Best
Mike

3 Likes

How are you viewing the data output during the pause? For me it seems to just hold mid-execution and waits to continue the execution instead of ending the loop execution and allowing you to review the output or edit. I tend to place a row / iteration / conditional filter (which has a bypass setting for normal use) on the underlying data. This sets a hard loop end point which allows you to execute through the end of the loop for review and editing. I also use a flow variable to stop the downstream execution below the loop end when it is not in bypass mode. It requires 2 connected components, but works well.

Hi @mwiegand , I’ve also run into this situation in the past, which you may also be interested in.

My solution was using a Java node in a “breakpoint” component and a disconnected “controller component” day tells it when to break.

I’ve actually been putting together a video demonstration of this which I plan to upload at some point.

I am not seeing the advantage of breaking the loop before it finishes, but that may just be caused by my own personal preference / use cases… My approach of filtering the data to the desired stop point earlier in the flow, and then placing the flow variable controlled breakpoint component downstream of the loop end allows the loop to complete for easy review of the output.

Of course I use parts of your breakpoint component @takbb in my loop editors as well. :wink: Why build from scratch when you have already made every component! :rofl:

Hi @iCFO,

My own use case is that I have a large loop and I know “something happens” at around about iteration 200, which has a knock on effect that I haven’t yet worked out, so I want to execute through the first 199 iterations, and then step each iteration to view what’s occurring. But you are right that there are other approaches. I’m used to the idea of programming language breakpoints in IDEs where breaking at a particular iteration and then stepping through is a common way of working.

1 Like

If you are diagnosing an issue that is caused by the cumulative effect of prior iterations, I see the advantage of being able to run the full data through the loop until the breakpoint and then stepping through each iteration afterwards. It was indeed the situation where I just had not encountered a use case that required a full run of all prior iterations before step processing through loop sequences.

That being said, I also see the advantage of my approach in for my use cases in that I can limit the size of loop runs however I want to narrow down on issues / reduce run times / quickly review completed loop data outputs. For instance… I could filter to run iterations 37-43, loop through as individual steps, and complete the loop at 43 for easy review of issues in the output table.

I will say, that I am typically doing more straight forward business cases where the cumulative impact of calculations is likely more intuitive and straightforward than something more esoteric like iterative statistical analysis.

1 Like

lol, mine is generally not esoteric, it’s more me just puzzling over my seemingly infinite ability to write “don’t-quite-workflows” and a spattering of Heisenberg’s uncertainty principle thrown in for good measure :wink: i.e Failing to understand why it breaks at iteration 200ish but for some reason if I filter out all the earlier rows, the problem goes away!

1 Like

I applaud the modesty from a sensei master! :martial_arts_uniform: You can now add “don’t quite work-flows” to the long list of your intellectual properties that I have coopted…

You can also go back to optimizing real time rocket re-entry trajectories based on hundreds of shuttle variables. :wink: :rocket:

I would really like to see an execute “x number of steps” to be built into the loop execution controls. That would build this into the platform and really simplify loop test / editing. I remember voting on it years back, but not sure if it still has an active ticket…

1 Like

@iCFO , there really does need to be a :joy: reaction as well as :heart: available on the forum! :rofl::rofl::rofl:

1 Like

Power Tools has a treasure trove of awesome capabilities! I still need to dig into settings / options and test everything, but it looks like it could replace a minimum of 5 separate programs that are always running on my PCs. Thanks a ton for passing this on! I feel like we need a separate forum area for passing along tips like this and discussion of hardware / related software.