Hello, My workflow uses a parameter optimization loop to find an optimal set of parameter values, which are then fed to a different node to create a final model. Simple enough, but occasionally the optimization loop end throws an error that not enough candidates were found, etc. and the entire workflow stops. In that case, I want to run an essentially identical optimization search except using a larger number of warm up rounds. I’ve tried numerous combinations of Try (Data Port) -> optimization loop node -> Catch Errors (Data Ports), but my attempted setup always has some type of error and doesn’t run properly (or often at all). I’ve cut-n-pasted the node or inside components to hard-code the higher number of warm up rounds that I then connect to the second Catch Errors port, with no success. Can somebody tell me how the setup should look? I suspect that will be easier than providing all the examples of what I’ve already tried as I’ve probably been doing something fundamentally wrong. [Loop%20structure] Above is the node I’ve referenced; the inside view of it is below. [Loop%20detail] Thanks in advance for the help.

Do you have a screenshot of when the loop fails and what it looks like and what can be seen as a result? If it is an empty table the empty table switch might be an option: [image] Empty Table Switch Sends input data to first output if and only if it is not empty. If there are no rows in the input data the second output is activated (with the empty input tab… Even better would be a sample workflow that demonstrates the behaviour.

I’ve tried many version of this, but here is the ideal version of what the workflow would look like structurally: [Ideal2] The failure message is: ERROR Parameter Optimization Loop End 0:1330:1339:1096:0:1003 Execute failed: The group of parameter combinations with good scores or the one with bad scores does not have enough members. Selecting a higher number of warm-up rounds or a more balanced gamma should resolve this issue. Once the top “Optimization Loop” fails, the second keeps running. Once it is complete, Node 1340 (Catch Errors) just turns yellow. There are no further warning or error messages, the workflow just stops. … If this isn’t helpful enough, I can build a sample. My hope was that the setup was glaringly wrong and a fix would be obvious to someone more knowledgeable than me, but perhaps that won’t be the case. If so, I’ll strip down my workflow and upload it.

I think this may be related to How can we use "try-catch" with Wrapped meta node? Perhaps a workaround might be to use a plain meta node? i.e. not a wrapped meta node?

actually I think it would help if we had a sample workflow that fails like you described it and we could see how to overcome this - meaning to active the alternative path.

I had previously tried using Metanodes instead of a Component node, but it is just a different error (see below since I just retried it). I also skipped using a Metanod/Component node completely and hooked up the Try/Catch to the expanded version of the nodes, also without success. I’ll pull out the confidential aspects of my source data later today and post the workflow so everybody can see the problem directly. Unhandled event loop exception. java.lang.NullPointerException at org.knime.workbench.editor2.editparts.NodeAnnotationEditPart.lambda$0(NodeAnnotationEditPart.java:101)* at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:37)* at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:182)* at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4213)* at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3820)* at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$5.run(PartRenderingEngine.java:1150)* at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:336)* at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:1039)* at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:153)* at org.eclipse.ui.internal.Workbench.lambda$3(Workbench.java:680)* at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:336)* at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:594)* at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:148)* at org.knime.product.rcp.KNIMEApplication.start(KNIMEApplication.java:149)* at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)* at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:134)* at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)* at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:388)* at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:243)* at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)* at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)* at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)* at java.lang.reflect.Method.invoke(Method.java:498)* at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:653)* at org.eclipse.equinox.launcher.Main.basicRun(Main.java:590)* at org.eclipse.equinox.launcher.Main.run(Main.java:1499)*

Here is the workflow itself (too large to upload directly). VolTrader - simp The error message was: ERROR Parameter Optimization Loop End 0:1330:1339:1096:0:1003 Execute failed: The group of parameter combinations with good scores or the one with bad scores does not have enough members. Selecting a higher number of warm-up rounds or a more balanced gamma should resolve this issue. ERROR Parameter Optimization Loop End 0:1330:1339:1096:0:1003 Execute failed: The group of parameter combinations with good scores or the one with bad scores does not have enough members. Selecting a higher number of warm-up rounds or a more balanced gamma should resolve this issue. Thanks in advance, again, for looking at this.

Hi @cybrkup , I looked into workflow to find out why the parameter optimization loop is actually failing. You have set the number of warm-up rounds to 7, this is a small number, especially since your parameter search space is quite big (4 parameters with big ranges and many steps). To get some meaningful results with Bayesian Optimization in this case, you should increase the number of warm-up rounds and the number of max. iterations. This will also prevent the nodes to fail anymore. Hope this helps and you won’t need to catch any errors anymore. Regards, Simon

Hi @SimonS , Yes, that is done intentionally to trigger the error for debug purposes. In “real” use I have the warm-up rounds set to 250 or higher, but occasionally still there won’t be enough viable candidates and the node will error-out. At that point I want (need) it to switch to much higher number of candidates or use Hillclimbing as the alternate node.

OK, so constructed a workaround that could catch the error. I stripped the (very large) example down to a core Metanode where the error occurs. I built an error catcher around that has an alternative table which is deliberately empty but has the same structure as the result. [image] Outside the loop there now is ab End IF node that would take the populated result but insights that both inputs have the same structure. The result from ‘Component A’ is empty due to the caught error so Component B will be chosen. [image] kn_forum_vol_trader_simple.knwf (598.3 KB) [image] catch the error from a parameter optimization loop – mlauber71 catch the error from a parameter optimization loop and replace it with a different result – Edit: KNIME — Cases, Switches and Catching Errors | by Markus Lauber | Low Code for Data Science | Medium

OK I streamlined it some more, just using the result of Component B as an alternative if Component A does fail [image] [image] [image] catch the error from a parameter optimization loop – mlauber71 catch the error from a parameter optimization loop and replace it with a different result

KNIME Forum Archive

Error handling (Try - Catch Error) Design Question

KNIME Analytics Platform

mlauber71 December 18, 2019, 7:03pm 11

OK I streamlined it some more, just using the result of Component B as an alternative if Component A does fail

Understanding Problem Error Handling Loop