Please take a look at the output node of these components. Even though the nodes prior execute and complete as expected, the final component output continues to cycle and never complete execution.
FYI - I witnessed this behavior in version 5.1 nightly, but haven’t tested it on the official release yet. I’m still working through issues before I can upgrade my working program to standard 5.1
Edit - I believe that all of these components had a Garbage Collector node attached to the variable output of the Component Output. My guess is that I just need to move it earlier in the process for it to work in the updated version. I will test later.
I thought that it was solved, but I was wrong. Version 5.1 still seems to think that the loop is incomplete on these component designs, even though it functioned fine in prior versions. Please take a look when you get the chance.
Just replying to make sure that you saw my edit. There definitely seems to be a difference between prior versions and 5.1 as to whether a component loop is complete or not.
I did a few test components and the issue seems to be that 5.1 no longer handles my approach of cycling through a component execution using a looped log file process. It doesn’t cause errors for “branching out of a loop”, but it does cause the component output to cycle endlessly.
In KNIME 5.1 “Component Output” nodes will turn executed/green only if all contained nodes in that component are executed. If there are nodes still executing the output node will also remain executing. Once all nodes are successfully executed the output node will also execute (green traffic light). If any one node fails execution, the output node will also be marked as failure (the error message on the node should give a hint which node(s) cause trouble).
Looking at your component I’m not sure what it’s supposed to do. There is a loop inside of it, whereby the Component Output node is part of the loop body itself (not directly, but it’s a side-branch of the body). What’s the designated output of the component? The result of the last execution?
Not sure what to do with it. I have a technical explanation for the problem. To me it looks like clashing contexts (component-start → loop start → component-output → loop end — no proper nesting) and KNIME should have not allowed designing these types of workflows from the start.
You can view the functional component in version 4.7.4 if it is helpful to see what it is doing. Essentially, it is creating a log file (if one is not already present) which is then used to save user changes to the interactive table. It then reads back the log file earlier in the workflow so that it updates into the interactive table to reflect the user changes (showing recorded changes in a different column and clearing the user interaction dropdown column for the next round). There are additional routings for writing to (and recovering from) backup log files.
The top output passes the altered input table, the bottom output contains a summary of the changes applied which can be used to sync settings between components in other parts of a workflow.
The function of the counting loop that is causing this issue is to trigger a second refresh after the user hits the manual refresh button. The first manual refresh button writes the users changes into the log file. The counting loop is just triggering a second read of that log file with the newly recorded user changes to update the interactive table and prepare it for the next user interaction. Up until version 5.1, I used this method to refresh the entire component using the garbage collector node to cleanup between cycles.
I think I have tried just about every trick in the book to attempt to close that loop on the alternate branches using flow variables. It doesn’t seem to matter how I try to block that first iteration from the component output (inactive, failed) or where I move that variable loop end (including before the output branches split), it still executes the component output and leaves the variable loop end node in a “queued” state.
Here is a dramatically simplified version that shows the same loop approach working cleanly outside of a component, but cycling endlessly when placed inside a component.
I modified the large workflow (which is huge…!) by rewiring the loop end to be properly inside of the component. Can you have a look?
(Edit: Workflow removed due to size constraints)
Specifically, what I did:
Removed the Garbage Collection nodes (didn’t have the extension installed - so it’s really only cosmetics)
Removed the “Variable Loop End”
Inserted a new “Loop End” immediately before the output node
You would still need to do some filtering (since you are only interested in the results of the last loop iteration but that should be simple).
Can you check if that goes into the right direction? (It’s the left-most “Header Re-Map” workflow.)
I’d like to note (again) that I believe the component had some conceptional flaws from the get-go since it has badly nested “scopes”. KNIME should have complained about it earlier - I’m going to open a ticket to improve error messaging here!
Thanks. @wiswedel I will take a look now. I am definitely aware that I was operating in the “grey area” of loop construction".
Did you see my simplified example workflow attached above? It shows a much cleaner example of a loop approach working fine outside of a component, but cycling endlessly when wrapped inside a component.
I was so fixated on goal of looping through the table writer / table reader that I never thought of the opposite approach of using a dual loop end on the outputs and connecting the table writer via flow variable.
I still think that the simplified workflow that I uploaded is worth taking a look at since it highlights that a loop structure can currently function at the workflow level, but fail to complete at the component level. Whether a loop structure works or not, I would expected the flow to execute and complete the same way inside or outside of a component.
Now the Column Expression node is no longer part of the loop. In versions prior KNIME 5.1 this node is only executed once after the loop has finished. In the current KNIME 5.1.0 release this workflow will run endlessly (which itself is not good and worth fixing … but in any case it does not complete, which from a pure workflow perspective feels right).
I hope that helps a bit with the clarification. I will bring it up with some more people from the team to discuss if we need to insert switches to allow for this behavior (and if, then only to guarantee backwards compatibility).
Thanks a ton for the explanation. I get it now, and I will easily be able to avoid it going forward. It may be worth looking at for compatibility as I am sure it will arise again as an issue for other users, but you probably have a pretty full list with the new UI development. I switched the solution to your last post, as it is a nice clean explanation of the underlying issue I was struggling with that will help other users.
I don’t seem to have title edit permissions in the new forum structure. You may want to re-title this thread to add “loop” and a better description of the issue to help draw users to it in the future.
That was the last piece of the puzzle that allowed me to switch my working installs over to v5.1. Thanks again!
