Any way to continue a loop "on error"?

I am using the Line Reader node within a loop to retrieve and process a list of URLs.  If the node receives a HTTP error, the script stops.  Is there a way to set it such that the script will just move on to the next item in the loop and not screech to a halt?  I have searched high and low for the answer the this but did not come up with anything helpful.  Is there someway to get to a setting on the node along the lines of "Continue Script On Error"?  Thanks very much in advance.

Hi,

we are working on adding such error handling capabilites to KNIME workflows. Right now you will need to have some sort of a node return e.g. an empty table indicating "failure" and then use the switches (for instance the empty table replacer) to run alternate paths of your workflow. But actually failing nodes (error during execution) will terminate the execution.

Michael

1 Like

Following up on this thread - any advice how how best to implement the equivalent of a java TRY {} CATCH block to capture failing nodes and triage ? I'm looking to run the workflows in batch mode via command line and would like to be able to handle node filure exceptions automatically if possible

thank you

Coming back to the original post, I think this can be implemented using the Java IF (Table) node together with the Java Edit Variable node which validates the URL (using Java code) and generates a variable "failed". This is used in the Java IF (Table) node which bypassed the Line Reader if "failed==true"; use the End Switch to bring the active/inactive branches together again. Give it try.

Thomas,

 

I tried something like this and it failed - failing nodes always drag the loop into the abyss with them. :-( Annoying since I have to parse a bundle of would-be PDFs which don't always compute for the reader node...

 

Cheers

E

Hi,

is there any solution to this problem at this time?

Thank you,

Josip

There are now try/catch nodes - put the node which files between a try and a catch node, and configure according to what you want to happen

Steve

Is there any documentation on how to make this work?

I've added a test workflow: the idea is simple, I have a table that gives me a list of things that need to be processed. In my example workflow it is either a calculation or reading a file.

If the calculation fails or the file does not exist, of course the node fails. 

At the end of the loop, I just want my list with an extra column that says whether an error occurred, and another column with, in case it did not fail, the result of the calculation or action on the file.

But the workflow simply fails, in both cases. The error is not caught.

So how do you get this to properly catch errors? 

Tim

The issue is the try/catch nodes work on inactive ports, not on errors. When an error happens and the node is not "compiled!", then the port is neither active nor inactive. Therefore to make the try/catch nodes work, you need to identify what is causing the target's failure to compile, and trap that explicitly. 

In the case of the FileReader, ths is because the file is not present. Therefore what I've done is to modify your top branch where the file is to be read in the following way:

1. extract the parent folder of the file, this may be done by converting the filename to the URI, and then converting the URI to a file path

2. list the files in the parent folder

3. do a row filter one the content of the parent folder for the target read file

If the file is not present, then this will result in an empty output from the row filter node. The try/catch can then be made to work in conjunction with an Empty Table Switch to read the file if the file is present, or to your active branch inverter if it is not. I have to kick the active branch inverter into life by forcing a failure (inactive port) using a Table Row to Variable node.

The other part you need to check in the Catch Error (Var Ports) node the "Always populate error variable", and to achieve what you are after, select in the Variable Loop End the specific variables. I've picked "FailingNodeMessage", "currentIteration", "a", "b", "filename" as examples.

 

The second branch is easier, As the error is in the java (division by zero effectively),  you need a try {} catch {} in the java itself to deal with the exception

try { 

      out_result = v_a/v_b;

      out_success = 1;

} catch (Exception e) {

     out_result = 0;     

     out_success = 0;

}

 

Note that although you can set out_result  to null, the node itself fails if you do this, therefore I've created a "success" variable to identify which of the try/catch in the java branch was processed through.

Hope it helps.

 

David

 

David,

thanks for your reply. Your first line explains it all:

The issue is the try/catch nodes work on inactive ports, not on errors.

The example I provided is just a trivial example. In real life, the loop runs over a lot of files, and the actions it performs are not trivial. So in case anything goes wrong on one of the files, I just want the loop to continue processing the other files.

Basically, what you're saying is that a real try/catch is not possible in Knime.

Tim 

For a purely generic error catch, it most likely won't work. However, if you can explain further what you are trying to catch, I can see if I've done anything similar. For example, if columns are different, I use a reference column filter to check, and so on to propagate an inactive port. Try/catch nodes can be used if you are willing to do a bit more planning.

David