Questions about the Try - Catch Errors node

[JaeHwanChoi]

Hello, KNIME Support.

I have a question about the Try - Catch Errors node.

I am currently using multiple Pyspark connections in connection with Livy.
If any of the multiple Pysparks give an error, I need to terminate the Spark Session, and if it doesn’t give an error, I need to go on to the next Flow.

I designed the WorkFlow below, but I have a problem with the flow that terminates the Spark session even if the Pyspark does not fail. I think the Catch Errors node is a node that is executed only when there is an error, but it seems that it rides the Flow even if there is no error.

Which part should I modify?

Here is the workflow I want to achieve

1. run Pyspark Script_1 and Script_2, and if either of them fails, terminate the Spark Session via the “Destory Spark Context” node.

2. if both Pyspark Script_1 and Script_2 are executed successfully, Script_3 is executed.

Any help would be greatly appreciated.

[AlexanderFillbrunn]

Hi,
I think you can do it like this:

The trick is the Active Branch Inverter. Because the Try node deactivates the branch with the error, the inverter makes sure the Java Snippet in my case is executed. In your case, you would connect the Destroy Spark Context here.
Kind regards,
Alexander

[JaeHwanChoi]

Thank you for your reply. @AlexanderFillbrunn
I would like to ask an additional question.

I will test the Spark Session termination method you suggested above, but one additional question I would like to ask is that after the Pyspark Script passes with success, if there is an error during the preprocessing to the flow variable, I want to catch the error with a try - catch node and terminate the work in the backend.

In Container Input(Variable), I want to create variables named “path” and “path_detail” and in String Manipulation(Variable) in the backend, I want to create a new variable as /path/path_datil.

If the String Manipulation(Variable) node fails because the Container Input(Variable) contains the variable “path_1” instead of “path”, we want to pass it through the Catch Errors(Var Ports) node and proceed with the remaining work in the backend.

If the Container Input(Variable) node gets an invalid argument, the String Manipulation(Variable) node won’t run at all, so we can’t catch the error with the try - catch node. Which approach should we use in this case?

Your help would be greatly appreciated.

[AlexanderFillbrunn]

Hi,
In that case it should be possible to wrap the node throwing the config error into a component. You can of course also put everything between try and catch into it. A component is treated like a node, but the configuration of the inner nodes happens at runtime and not configure time of the component, so you can then catch the failure of the component when the flow variable causes an error.
Kind regards
Alexander

[JaeHwanChoi]

Thank you for your reply. @AlexanderFillbrunn

Can you explain this in more detail? Or can you provide an example workflow, it’s too hard to understand from your description.

Thank you.

[AlexanderFillbrunn]

Hi,
Please have a look at the following workflow: TryCatch Config Error Workaround – KNIME Community Hub.
When I run it, it looks like this:

Whenever an input to a node or its settings change, it is configured, i.e. settings and inputs are checked and information what will be produced when the node is executed is passed on to subsequent nodes, which are configured as well. This allows you to select columns even when previous nodes have not been executed and have therefore also not produced any data yet. Now at the top of the workflow, the Java Edit node throws an error when checking the configuration. This is not caught by the Try-Catch block. At the bottom, the configuration step of the Java Edit node is performed only when the outer component is executed, so to the Try-Catch it looks like the execution and not the configuration of the component failed, allowing it to catch the error.
I hope this helps!
Kind regards,
Alexander

[JaeHwanChoi]

Thank you very much for your response. @AlexanderFillbrunn

I have one more question.
I am trying to run a WorkFlow on KNIME Server that catches errors.

The job history shows a green light as a success when the Workflow succeeds with the Catch Errors node because a certain node failed due to an Error. (Picture below)

However, what I want to do is
I want to leave the Job history as “Fail” even though the Catch Errors node successfully completed the Workflow because the node failed with an Error.

Is this something that can be implemented in the

Any help would be greatly appreciated.

[AlexanderFillbrunn]

Hi,
You can trigger a failure by using the Breakpoint node. This node triggers an error based on a condition you can control, and you can also choose the error message.
Kind regards,
Alexander