Hi,
I have questions regarding the ‘Log Parser and Usage Reporting’ workflow:
What is the difference between the 'Execute workflow ’ and ‘WebPortal execution’? Why do you differentiate them as both relates to execution of a workflow on the web portal?
To filter workflows, you use a lag column node followed by a rule-based row filter which selects workflows only if execute is preceded by load. This will led to not counting workflows when this condition is not met. This is the case when many users run different workflows on the web portal in the same time windows. In the knime table, entries for the same workflow execution will not be necessary consecutives; this workflow will not be selected in that case. I have noticed this when executing the workflow on our server. I have also noticed that the lag column node may create entries in the lag-1 col that does not refer to the same workflow. Would you have any suggestion to fix these filtering issues?
“Execute workflow” refers to jobs that have been started from either AP client or via REST, “WebPortal execution” means that a job was started from WebPortal. The log messages in both cases are slightly different, so we are able to tell them apart.
You make a very valid point here. In theory, the action of executing a workflow is always preceded by the action of loading the workflow. However, it is possible that in the (very small) timeframe between loading and executing a job, another user starts a job as well. In those cases, there will be a mismatch unfortunately. However, note that there is very little lag between loading and executing, usually both occur within the same second (you can verify this in the logs themselves). So in practice, I would say the results are reliable.
Note also that the log parser workflow is best suited for analyzing events on a longer time scale. If you want a more “reliable” way to track server usage, I recommend to set up a workflow that regularly queries the :jobs endpoint of the REST API, and then store those results in a file or database.