Solutions to "Just KNIME It!" Challenge 17 - Season 2

:boom: New Wednesday, new Just KNIME It! challenge! :boom:

This week we’ll tackle an important topic for the first time! :exploding_head: :ok_hand: Best practices while developing workflows: error handling and logging :spiral_notepad:. These are two important strategies that improve the quality and robustness of any solution, but they are often forgotten… Well, not this week! :smile:

Here is the challenge. Let’s use this thread to post our solutions to it, which should be uploaded to your public KNIME Hub spaces with tag JKISeason2-17.

:sos: Need help with tags? To add tag JKISeason2-17 to your workflow, go to the description panel on the right in KNIME Analytics Platform, click the pencil to edit it, and you will see the option for adding tags right there. :slight_smile: Let us know if you have any problems!

1 Like

Hi everyone,
here is my solution. I use “Just Knime it challenge 10” workflow.
I can not handle file I/O errors (i.e no file to read) with try-catch nodes.
I write the log messages to the “/data” file of the project.

4 Likes

Here’s as close as I could get. Used Excel with separate tabs for successes and failures.

2 Likes

Hi all,
This is my solution.
Breakpoint node was used to generate an error.
The workflow for which you want to perform error handling and logging can be used by placing it in the green area and connecting the flow variables back and forth.

3 Likes

I submitted Ver1 and after learning about @sryu , I found that there were logical issues with my workflow, so I deleted it. The problem with that version is that there are multiple entry nodes and the time is not accurate to milliseconds.

2 Likes

Hi Everyone,
Lot’s of fun doing this one. Last fall at the KNIME UGM in Austin, i learned this technique from @alinebessa in the course that she taught. In practice, i have been using it ever since in my real world applications and i can say it really makes them very robust.
For this challenge, I actually struggled to create an example workflow that threw the proper error. Also what was very exciting is that i finally installed v5.1 of the KNIME AP and for the first time used the new UI. it is gorgeous!!! But i admit, when i had trouble i reverted to the old UI to get things done…
To the solution:

The Orange Workflow. A hypothetical workflow that is not intended to solve any problem, but has operations i use all the time. This workflow starts with a list of produce being purchased at the grocery store. Each row is a different customer and each column is contains a separate item of produce for that customer. The rows are then looped over and inside the loop, it is necessary to use each customers list of produce to rename the rowID (again… this is totally hypothetical). An action is the added with the java snippet node. When the loop is finished, you have a final table with the list of items and the action. Here are the before and after tables:

image

image

The Green Workflow. In this modification, customer = Row2 decides to buy two bananas.

image

This creates a problem later in the original workflow because you cannot have two rows with the same RowID. This issue throws and error that we deal with using the Try-Catch approach.

image

On the top stream of the Try-Catch, the RowID node throws the error and so the error-handling bottom stream deals with the issue by appending the row index to the duplicate entries in order to uniquify them. The error is circumvented and the loop can continue until execution is complete.

image

The Blue Workflow. This last version handles the error logging. I dont normally do this so there could be a better way for some parts. After the Try-Catch segment, i added a metanode that adds a few fields using Java Snippet (Simple) nodes, in part, via flow variables created by an Extract Context Properties node. Also included were Java Snippet (Simple) nodes for the Failing Node and Error Message provided by the Catch Node. Be sure to select “always populate error variable” in the Catch configuration dialog, otherwise the Java Snippet (simple) node will throw an error (Fun Fact- you can overcome this error using a second Try-Catch sequence :laughing: :rofl: :joy:)

Finally, I add a timestamp at the end of the workflow to indicate when the workflow finished… since it executes so fast, i inserted a Wait node set to a 5 sec delay just so the time stamps were actually different.

Finally we end with a csv writer node as requested. In order to properly log new entries on subserquent execution, be sure to select append in the CSV Writer node configiration dialog.

Hope you had as much fun as i did… if anyone sees a better way to do any part, please let me know. I am always excited to learn from you all.
L

4 Likes

Hello everyone,
Here’s my solution.


For this challenge, I prepared the following example workflow and files. The files used in this workflow are in the current workflow data area. It was difficult for me to come up with examples that cause errors. Thanks.

[Workflow]
Calculating the sum and mean of employee’s working hours per week. Three files are processed one by one using the “Table row to variable loop node”.

[Files]
The three files containing employee’s working days and hours, one of which contains an incorrect value, causing an error in the “String to Date&Time node”.
image

3 Likes

Exceptions often occur, and I used a simple string format to date conversion error to simulate exceptions, which can also be manually changed to normal.
The main demonstration part of the workflow: The Try/Catch structure and logs are basically consistent with the previous colleagues’ workflow. Using the Try (Variable Ports) node can make exception capture work in the control flow, without intervening in the data flow of the main workflow branch, which may be easier to handle.

2 Likes

Hello everyone,

Here is my solution. I took a workflow from JK2-10 and upgraded it according to the requirements. I also used Integrated deployment nodes, so it follows the full cycle of development-deployment. I also added the tests in the same workflow so one can easily make sure that errors are properly caught and logged.

2 Likes

Hi, KNINJAS :ninja:
I hope you are enjoying your summer :beach_umbrella: :smiling_face:

Here is my solution: JKISeason2-17 Ángel Molina – KNIME Community Hub

I am really happy this week because I have finally been able to put into practice a real problem that I have been facing for the last few weeks. When I was trying to find the geometry of a country or region, many times the node failed because the region doesn’t exist or the name is not standard. For this reason, this challenge has come to me like a glove.

See you next week! :mechanical_arm: :orange_heart:

4 Likes

My try on the error handling … Need to error check the error handling flow itself and for sure need to learn more of side .

2 Likes

Hi Everyone :slight_smile:

Here is my solution for this week:

I’ve used the -Single Selection Widget- to allow the user to decide if they want to simulate a Success or a Failure. This is followed by a -Case Switch- that switches between using correct date data or incorrect date data:

When the -Try and Catch- is used, the correct date data is successfully run through a couple of -Date&Time- nodes. However, when the incorrect date data is used, the -Date&Time- nodes are bypassed and instead a -Constant Value Column- node is used to assign a status of “Failure”:

After the -Try and Catch-, the -Extract Context Properties- is used and an execution time is generated:

The information is then written to a CSV, which is updated each time the workflow is run by appending the new row of data.

You can find my workflow on the hub here:

Best wishes
Heather

5 Likes

:fire: As always on Tuesdays, here’s our solution to last week’s challenge! :fire:

:heart: I personally loved this challenge because I’m a big fan of error handling and best workflow practices in general. They really help when you’re building larger projects with more people! :question: Shall we have a challenge on testing soon??

:sparkles: See you tomorrow for a new challenge!

5 Likes

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.