How to retrieve loop index of failed request

KNIME Analytics Platform
1

Hey guys,

so I have a table with multiple characters that I concatenate to urls. Then,
I want to loop over each url and try to execute it via a GET Request.

If one of the requests fails, the program should stop and I want to know the index where it has stopped.

image

However, when there is a failure, the catch block always shows this:
image

My hope was that the second last row (Row0) indicates where the error occurs. However, this field remains constant, no matter if the first or 10th urls was wrong.

Any ideas?

2

Hi @ThoMi , it’s not really possible to comment on the “Row0” entry as we don’t know what that flow variable is called, or what it represents.

Does your metanode always produce a single row, with the rowid “Row0” for example?

You mention that you are wanting to loop over each url in the table, but there is no indication in the screenshot that you are looping. So is the GET Request being provided with one URL at a time, by a flow variable, or all URLs in the table using a column name?

I think that the GET Request is designed to attempt multiple requests and either it is set to fail completely if an error is returned, or else it is set to continue and simply report the status of any failing requests.

Possibly neither of these work as you want for your use case.

Here is a suggestion that may be of use.

In the attached demo workflow, the GET Request is configured to report back the status of each request, but is not set to fail.

image
image924Ă—878 78.7 KB

I have therefore not put it inside a try catch block for the purposes of this demonstration. Instead, additional nodes in the red box below are there to check on the returned status. If any failure is found, it reports back the first row that fails and halts execution.

image
image1866Ă—748 114 KB

This show-cases the use of a couple of nodes in particular.

Firstly, the Node Annotator which has recently become one of my favourite extension nodes. It is freely available from NodePit, by installing the NodePit Power Nodes

Big shout out, and kudos to @qqilihq and @danielesser for providing us with these fantastic nodes. Thanks guys :slight_smile:

The second is the use of Java Snippet to halt execution with an error message..

Coding java snippet to throw an Abort halts your workflow. It can be useful occasionally.

	throw new Abort("URL Failure RowId: " + v_FailingRowID + " "+
	c_url+ " reason : " + v_Status + "\n" +v_ErrorCause);

An alternative extension node that can also be used to halt execution if you don’t want to write the java statement is Fail in Execution, part of the KNIME Testing Framework UI extension, and also available from here:

When provided with a set of “good” URLs the workflow completes, as shown in the image I added earlier.

But look what happens when it is supplied with at least one bad url:

Not only does the workflow abort at the java snippet, but the “hover over” message on java snippet details the first problem found, and the Node Annotator displays details right there in the workflow!

image
image2132Ă—747 141 KB

How cool is that! :sunglasses:

This is the config for the Node Annotator where you can just include flow variables right in the message text, so it’s super easy to use.

image
image760Ă—717 80 KB

OK, so I had a bit of fun with the above, but I hope it gives some ideas on a possible way forward.

Find failed GET request.knwf (18.7 KB)

3 Likes
3

Always a pleasure! :smiling_face_with_three_hearts: – and a huge “thank you” for the awesome post with explanations and examples.

2 Likes
4

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.