How can I generate a log from an existing process, that contains information such as which block, with what variables and values, in what time/duration has been executed?
Hi @zahrasadeghib ,
Could you please explain what you mean by existing process?
Do you mean a running workflow (a job)?
Regards,
Temesgen
2 Likes
Hi,
By existing process I mean any running workflow in Knime, for instance, the following workflow
Thnaks in advance,
You can collect some info about your workflow execution in a table and write it to a file. Here’s a mockup of how that would look like. month is a configurable Flow Variable, representing workflow settings:
log WF info.knwf (35.5 KB)
Hi,
I want to get the execution log from any existing workflow, as is, without modifying it.
So for example, some else might have provided a large complex workflow, and I want to get an execution log from it. modifying it may not be a feasible option.
For example for the following workflow
The parameters part could be any information about what did happen in the blocks.
Hi,
What you are looking for is NOT currently possible. The log entries which are saved under knime.log are global at the AP level and there is NO workflow specific information attached to them. There are also log entries completly unrelated to any workflow. If multiple workflows are running in parallel, order can NOT be guaranteed.
If saved, the executed workflows themselves are kind of self documenting.
May I ask what is the use case here?
Kind regards,
Temesgen
2 Likes
@temesgen-dadi is it possible to have workflow specific logs on the server like you could have on the analytics platform?
@mlauber71 AFAIK, it’s not possible to have workflow specific logs on the analytics platform either.
On the server, Error and Warning messages can be retrieved as shown below.
Regards,
Temesgen
Thanks @mlauber71! Embarrassingly, I didn’t know about that. I learned something new today 
Coming back to your question, yes it’s possible. I just tried it myself. The respective entry in the executor preference would be
/instance/org.knime.workbench.core/logging.logfile.location=true
Then the log will be saved under the job. You can, for example, save the job as a WF on the server - copy it to your local workflow repository and navigate to the WF directory to get to the log file.
Regards,
Temesgen
1 Like
Hi,
Thanks for your reply.
About my use case, I need the kind of log as I said, to perform process mining on that log. Process mining is a hot topic nowadays. It can discover the bottlenecks in a running workflow, it can easily give us insight about which path in a workflow takes more/less time, so that we can optimize the workflow.
I probably can change the source code of Knime to generate the kind of log I described.
Knime has a large codebase. Can you please let me know how can I get some hints about which part of the code is the best place to get started with this?
Thanks in advance,
Maybe the Timer Info Node helps you with that? It gives you number of executions and execution time for each node, not the settings though.
Here’s the output of a small WF I’m currently working on:
Reset: When a node is being reset, either from the user or loop nodes. This is a bit weird, since it means nodes inside loops will only show 1 execution in this column.
Start: When a Workflow has been opened.
Node 15 is a Metanode, so it won’t show up and the contained nodes additional IDs.
Additionally, if you turn on INFO level logging and enable per workflow logs, you will have lines as follows. Notice at the end there is the time it took to execute the Python Learner node. The resolution is in seconds but it’s better than nothing.
2022-01-19 09:36:01,596 : INFO : KNIME-Worker-52-Python Learner 0:0:0:105 : : LocalNodeExecutionJob : Python Learner : 0:0:0:105 : Python Learner 0:0:0:105 End execute (31 secs)
The log file is located inside the workflow directory under the name knime.log
2 Likes
I do so every day with KNIME
1 Like
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.