Loop failure, move to next batch

Hi!

I’m calculating money multiples (TVPI) in a loop workflow, and I have a question how to do basic error handling in looped workflows. TVPI calculation divides outgoing cashflows with incoming cashflows. In some cases, no outgoing cashflows have occured yet and the bottom part of the flow returns an empty table → causing the workflow to halt. Is there some way I can move to the next loop if this occurs?

Thanks for the help :=)

Hi @Data_consumer , in the screenshot it doesn’t appear that an empty table has been created, but simply that no request has yet been made to execute the while loop (as the nodes are all in the yellow state), but perhaps that image wasn’t supposed to be representative.

Which node does it actually fail on?

Which branches can cause problems and so potentially need to be “skipped”?

An empty table by itself should normally result in an empty table warning but not cause a failure by itself although subsequent processing can fail depending on what you are doing.

If it’s causing problems, there is the “empty table switch” node which is like an “if” or “case switch start” node. It executes its top output branch if table is not empty or the lower branch if it is empty. Join the alternative branches together again with a “case switch end” or and “end if”, when you’re passed the point where an error may occur.

Have a play and let is know if you need more help.

btw, There are also try-catch nodes which can help too depending on the failure but they can’t catch config errors (and a config error can unfortunately come about at runtime if generating flow variables to set options, but the flow variables are no longer valid, which can be a bit of a pain!)

1 Like

thanks @takbb! I will create a better explanation and come back with an example. :slight_smile:

example 3.xlsx (10.5 KB)
Calc mutiple - Export example.knwf (206.2 KB)
Hi @takbb Here is an example with 2 datasets. Company one fails, because it has no transactions to the distribution side (outgoing cashflow). Company 2 has both transactions and should run in the loop once the company 1 has failed.

Thanks again! :pray:

@takbb Tried to add the empty table switch and case switch end, but it just halts the loop from running the second batch :frowning:

Thanks!

Hi @Data_consumer

Tnx for sharing your wf. It makes it a lot easier to find a solution.
In this case I added the Try / Catch nodes Calc mutiple - Export example_V2.knwf (247.6 KB) , so your wf will continue even if it encounters an empty table.


The second thing I did was to check the "Allow changing table specifications in the Loop End node.
Schermafdruk van 2023-12-15 11-04-39
See if this works for you.

gr. Hans

5 Likes

Hi @Data_consumer ,

While writing, I see that @HansS has supplied a possible solution using Try Catch nodes which are certainly a possibility, and that’s good because there you have a good example of how that construct can be used.

I’ll continue with a discussion on how Empty Table switch could work, and you then have lots of options :slight_smile:

So… the Empty Table Switch…

… it depends on where you are adding the switch, and it’s possible I don’t know some other circumstances that we need to allow for, but it appeared that in this case it was the Top k Row Filter that didn’t like empty tables, so I added the switch just around that:

So in this case, when the table is empty, that branch is deactivated and the rest of the flow can continue, and it was able to complete

You may have other use cases that require slightly different action. I see there are other top K Row Filters, so possibly each of those would need a switch.

Alternative was to avoid executing the component altogether by putting an Empty Table Switch around it in the main workflow. This is marginally more involved :wink:

Now if you add the switch there, both loop ends are liable to fail because the table structure following the Column Appender will potentially differ on different iterations. The simple solution for that is the configure both of those Loop Ends to “Allow changing table specifications”

There is an alternative to allowing changing table specifications, and that would be to ensure that even when the component is skipped, the alternative branch supplies an empty table of the same type that the component would have created.

You could achieve this as follows:

As a one-off exercise add the following onto the end of the component:

The row filter should be set to exclude all rows

The table writer can be set to write to the workflow data area:

Execute that and you now have a blank version of the component’s output stored in a local file.

Discard those two nodes and modify the workflow as follows:

The Add Empty Rows should be configured to add “an additional zero rows”

In other words… it does nothing. It is there purely so that we can attach something to the lower data port of the empty table switch, and thus allows us to run a flow-control line to a Table Reader on this branch.
The Table Reader is then configured to read in the blank data table that we created earlier.

As a side note, the “Add Empty Rows” trick is what I used to use before installing the NoOp nodes from NodePit. These are a set of “NoOperation” nodes which are there to do nothing in circumstances like this. These are part of the “NodePit Power Nodes”, and I recommend installing them for cases such as this, as it is clearer what they are there for.

Anyway, hopefully with one or other of the above mods, you can make you workflow run to completion :slight_smile:

3 Likes

@takbb & @HansS You guys are AMAZING!!! Thank you so much for the help both of you :pray: It it possible to accept 2 solutions :D?

2 Likes

@Data_consumer , unfortunately only one solution can be accepted… but @HansS was first! :wink: … besides at “only” 235 accepted solutions, he needs the boost :rofl:

3 Likes

Thank you @takbb , you are too generous for this world :blush:.
My solution was just 3 nodes, and you needed a lot of nodes. I think that this number of nodes in combination with the clear explanation could also be appreciated with a solution :rofl:

4 Likes

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.