Hi,
I have a loop where I make an API call, which returns two datasets. Hence, I am using Loop End (2 Ports) to output both datasets. The API returns 2000 records per call based on timestamp.
I want the loop to stop when API returns no more records.
However I am not sure how I can end the loop dynamically. I am using Counting Loop Start node. Should I be using something else?
When there is no records, the loop end is giving an error: “Active Scope End node in inactive branch not allowed.”
Below is a screenshot of the wf. I also already tried try/catch with break point. Thank you!
Hi @rumanac , it’s not clear when your loop should end. Is it when both datasets are empty, or if any of the dataset is empty?
Also, I am not sure how you are using the Counting Loop Start. You would use this node if you know how may iteration you want to run. In this case, it sounds like you want to keep looping until a condition is met. In that case, the Generic Loop Start is better.
The problem though is that Loop End nodes (single port or 2 ports) does not accept conditions to end. This is control rather by the predefined number of iterations, such as Counting Loop Start (as you have, but does not look appropriate here), Chunk Loop Start, etc, anything that predefines the number of iterations.
In this type of situation, where the control to end the loop is the empty dataset(s), you would use the Generic Loop Start + Variable Condition Loop End. The problem is that you want to output 2 datasets, which is the tricky part, because only the Loop End (2 ports) outputs 2 datasets.
One way handle the 2 datasets is to concatenate them inside the loop, and then separate them outside the loop:
2 Likes
You could also just save the files in an intermediate table and load them again after the loop end. Maybe not the most elegant way.
One might have to toy around with the various loop concepts, switches and try/catch combinations that are there.
Hello @rumanac,
this topic might help as in it you’ll find nice tutorial about paginating API calls using recursive loop.
Only in your case you might need recursive loop end with 2(3) ports.
Br,
Ivan
Hi @mlauber71 , yes there are a few different ways to extract the data from the loop into 2 datasets. Just wanted to keep it simple for the first version, and also it was past midnight on my side when I replied 
Saving them in an intermediate table is definitely what I had in my mind, and I would probably do this in virtual tables via an H2 instance loaded in memory.
1 Like
Hey @ipazin , the recursive loop crossed my mind, but I thought one of the requirements was to get 2 datasets out of the loop. Though the Recursive Loop End (2 ports) does offer 2(3) ports as input, it still outputs only 1 port, hence why I did not go that way.
But usually for this type of operations which is similar to pagination, yes, the recursive loop would be the approach I’d use.
1 Like
H2 could indeed be an interesting option if speed is an issue
Hello @bruno29a,
indeed you are right. Forgot that
However dynamic ports are coming to loop end nodes with next release and are part of nightly build:
https://www.knime.com/nightly-build-changelog
Br,
Ivan
3 Likes
This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.