Is there any way to trap node errors to allow loop continuation?
Here's my scenario:
I have a data set containing all employee performance, and I'm looping through each individual employee to predict performance using several techniques for the purpose of understanding which technique works best for each individual. Invariably an error will occur at some point on some technique, and the message is always nebulous (ERROR Polynomial Regression Learner 9:407 Execute failed: Index: 4, Size: 4).
Typically I just remove this method so the loop can continue, but I'd prefer not to do that because it can be time consuming. I'd prefer to automate but I can't figure it out. Any ideas?
On a related note, is there a place where these types of errors are documented? Is there a guideline for understanding what the message is telling me? I'm experiencing them on various mining/regression learners and the messages usually indicate some index violation.
did you consider using the Try and Catch nodes for your scenario? They help to trap errors by executing an alternative path when that happens. There are few variations of them depending on the use case.
Regarding the interpretation of the errors from the learner/predictor nodes, you can usually get more details by increasing the logging level in the console (not recommended) and/or in the log file (recommended).
Here is how to do it:
---------------------------------------------
If you want to change the level of detail (either of the log file or of the console view) go to
File/Preferences.../KNIME for the level of detail of the log file or to
File/Preferences.../KNIME/KNIME GUI for the level of detail of the console view.
You can choose between:
DEBUG
Debug messages mainly used for development. It is not recommended to use this for the console view since it slows down KNIME.
INFO
Logs information messages. Not really important but also not completely useless!
WARNING
If a node fails in configure a warning message is also issued. Warning messages are not fatal; usually the workflow can continue to be executed but they denote that something worth to knowing about has taken place. Default and recommended level for the console view.
ERROR
Only issued when something fatal has happened, i.e. the workflow can no longer be executed.
-------------------------------------------------
In general those errors depend on your data (missing values, too few data points etc.), the node configuration (wrong parameters), or a combination of both.
If you don't spot your issues you can always post again in this forum to get support with it, but usually that requires sharing the faulty workflow and at least some test data (which you can always anonymize first if they contain sensitive information).
Thanks for your quick response. Is there an example that shows how to use try-catch with a learner node?
Regarding the error logging, it looks like default log level is error on gui and debug on file, which is what you're suggesting (I think). But I can't find the error logs. I've scoured the default directory but there's nothing that matches my workflow name and no files that match timestamps. Any guidance here?
please see the attached example I just created to illustrate how error handling works.
The learner is in between a Try and a Catch node. If the learner triggers an error, the result of the alternative branch is sent to the output instead of the one from the main branch.
In this example, to trigger the error in the learner, just remove any value from the X column in the input data to generate a missing value (and do not change the configuration of the learner to automatically handle missing values otherwise the error will not be triggered).
You will see that the learner triggers an error but the workflow executes to the end and the output is still valid as it comes from the fall-back branch.
Regarding the Log file, you can display it within KNIME by choosing View --> Open KNIME Log