Does Knime has capability to report workflow logs with the below information ?
- Workflow start time/end time
- Numer of records processed at each node
- Any other valid details
- Success Failure messages
A practical usecase would be to build a dashboard for Workflow monitoring.
I would like to load these details into a control table for audit purposes. Please help.
We are working at the moment on a feature where that kind of information can be exported as xml from a workflow. Besides the features you mentioned, this will also include node settings, information about the environment the workflow was run on, which plug-ins where installed, etc.
Thanks for the response. If the solution is in labs, when is it expected to release ? Until then do you think of any alternative ?
We expect it to come with the next release in July. Exporting an executed workflow and saving it gives you already a lot of information as the data is stored with the exported workflow. In addition, you can also save the log file (View->Open KNIME log) which gives you information about the error messages, in which nodes and when they occurred, etc.
Hope that helps,
Exporting each workflow everytime is not a feasible option and logs don’t give number of records processed at each nodes. What is the alternate solution available until july ?
Hi there @sudheerkopparapu,
did you think about implementing certain points in a workflow and then export that information to a file or even better to a database? Points 1 and 4 should be quite easy to do. Point 3 implementation depends on what valid details you need and that leaves point 2. Although not complex to implement for a single node adds complexity and execution time to whole workflow if you wish to have this kind of info for every node. I can imagine this being a requirement but not sure how valuable this information actually is
Could you pleas give me pointers on 1 and 4.
Regarding 2, we need checkpoints at very few nodes. So need pointers on this as well
Hi there @sudheerkopparapu,
For 1 you can use Date&Time Configuration node both when starting your workflow and ending it with option Use execution time checked. It will give you start and end timestamp in flow variables.
For 4 use Try/Catch sequences. If you workflow is small you can use only one and if bigger you can use one sequence for each logical part of your workflow. See this example.
For 2 you can use Extract Table Dimension node which will give you number of rows and columns both in table format and as flow variables. The name of node which processed those records needs to be hard-coded somehow (use String Configuration or simple Table Creator node.
Additionally you can use Extract Context Properties to get workflow name and have something like this as reporting records:
It requires a bit of effort but is most certainly achievable! Also there are topics where similar was discussed so give forum search a try and you might find additional valuable information!
This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.