my aim is to create an automated workflow documentation. Therefore, it would be nice to have all the node IDs of the workflow in a table in the EXACT order they are executed, together with the “custom description”. I can isolate the “custom description” from the settings.xml file and the node IDs from the knime.workflow and/or Workflow Summary using the XPath node but I failed to isolate the exact sequence of the nodes as they are executed in the workflow. I also tried the “Timer Info” node the same (even after sorting for execution time). A similar question was posted in the forum: Get the whole workflow flow information - #2 by Ema and How to write a documentation for a KNIME workflow? - #9 by Patrick1974
So my question…
Is there a way to isolate node IDs/Labels in the exact sequence of executed nodes of a workflow? E.g.:
The problem might be associated with the fact that my workflow is quite big with several branches and the node “composition” has changed over time.
The workflow “Interactive Workflow Metadata exploration” here (Interactive Workflow Metadata exploration – KNIME Hub) is pretty close … But still I cannot get the correct order displayed in the interactive table.
you can parse KNIME log to get exact sequence of executed nodes of a workflow as there is timestamp within it and messages like: 2021-06-11 13:28:57,898 ... Data Generator 0:19 has new state: EXECUTING ... 2021-06-11 13:28:58,952 ... Data Generator 0:19 has new state: EXECUTED
Additionally in workflow.knime file (located within workflow directory) there is information regarding connections between nodes so you can map branches to nodes if needed.
Just to be sure, with KNIME log you refer to the knime.log file in folder …\knime-workspace.metadata\knime. I am asking because when I open this file and search for similar entries (e.g. EXECUTUNG or EXECUTED) I cannot find anything besides warnings. Do I have to change the “level of monitoring” (e.g. WARN, ERRORS etc.) to increase the depth of information for the log file somewhere? And how do you differentiate between different workflows within the file (just by date and time?). I hae to admit I will have to have a deeper look at the file.
The good old KNIME question. Is there maybe an example workflow for parsing the KNIME log? So far I could only Google search find workflows for parsing server logs.
… especially the part “Replacing the first config-tag” was neccessary to parse all nodes but I was just wondering if there is a configuration of the XPATH node to do it without manipulation of the workflow.knime file.
Many thanks again.
I will give an update on far I have come with the new information.
regarding the possibility of fetching the required information from the workflow.knime file mentioned by @ipazin, you may be interested in the following demo workflow I’ve uploaded to the hub
The demo reads its own workflow.knime file, and interprets the xml to fetch the list of nodes, and the destination connection for each of them. It then “tree walks” that information by a recursive loop to determine the sequence for the execution of the nodes.
You can modify the “open file or folder” component to supply the absolute path to any worklflow.knime file. The output from this may be of assistance to you.
Hi @anjo, yes it does seem complex, but that’s simply because of the structure of the data it is intepreting from the XML file.
The workflow can be broken down into the following sections:
1 - read the XML file into a table
2 - derive the set of nodes
3 - derive the set of connections
4 - determine which nodes each node connects to (both in and out)
5 - walk the connections, to determine sequence
