Variable loop end node stuck at 2nd iteration

Hello,

I am extracting data from a db into monthly parquet files stored in a R2 bucket.


At the end of second iteration (#1), while the parquet is well at 100% upload, the variable loop end remains stuck in “queued” state and does not activate the next iteration.

Have been using this kind of sync in several workflows but it is the first time I have such an issue. Anybody experiencing the same issue?

Have seen other posts related to slowness of some end loop nodes, but I have kept it running for 2h without progress. So, it doesn’t seem a matter of slowness.

I am using Knime AP 5.4.4.

Regards,

Sébastien

PS: Here the content of the previous metanode:


Here is the state in which the workflow is stuck:

@szawadski can you give us more details about what this node is doing that is stuck. Also can you provide a fresh log in debug mode.

I found the issue…
I had a “dead branch” in the extract node. As that branch was remaining is queued state, although data was processed out of the metanode and made available to the next node (ultimately to the “Variable loop end” node), the loop end node was also not moving to the next iteration, waiting for the dead branch to finish…
I was fooled by the fact that all metanodes outputs were green and I didn’t think to look into previous nodes.
Is it an expected behaviour to take the worst state of all nodes within the loop (queued in this case), even is that node is not connected to the variable loop end?

@szawadski well. The nodes are connected to the loop start so they start. And if they would finish the loop would just start the next iteration. But if they are stuck the whole construct is stuck. I think best practise is to not have any orphaned nodes within loops as it might lead to confusion.

3 Likes

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.