The best way to docment Knime workflow steps

KNIME Development
#1

Hi: does Knime provide natively any API or utility to document the steps of work flow so we can generate a report with input and output data and the steps related to the whole work flow. If there is not any thing provided natively, can some shed some light what is the best starting point to achieve this functionality.

Thanks,
RJ.

#2

Hi rashid,

I'm not sure, what you expect by "documenting the steps of a work flow". I thought by definition KNIME is a great tool to document each step that is performed on your data, with the graphical representation of the workflow/pipeline. This pipeline is preserved when you save the workflow, with all settings and intermediate results.
There is no other, more text based documentation. And if you are thinking about analyzing the log file or the workflow.knime file, please be aware that its format and content may change at any time without notice.

- Peter.

#3

Hi; Peter,

Some times if you are performing some kind of analysis via work flow nodes you also required to generate a pdf or some kind of document for reporting purposes; so later some one can go through your steps and read your analysis and data related to analysis from a document. I guess what I am looking for is a mechanism where I take a whole work flow and generates a document that shows the steps of work flows with in and out data from each node. Note: step of work flows could be just high level description of the node algorithm in plain English; that can be read by the comments of the execute method of NodeMoel.

I am not sure how Knime stores work flow internally; is just the class files related to all nodes in a project directory; or there is some XML representation (meta data) of the whole work flow with input and output data to these nodes. I guess I have some idea about how to read the data after reading "How to save and load your internal representation" section of your extension documents.

Thanks,
RJ

#4

Hi: Peter,

Is there any way I got some more info about workflow.kime, setting, data and spec files and their inter relations and what kind of information resides in these files.

Thanks,
RJ

#5

Is there an easy way to get an ordered file of consisting on something like the node type, node name, user defined name and all of it's parameter settings?

#6

Hi Jay,
Hi RJ,

there is not really an easy way...but it is no secret, how KNIME workflows are stored. But please note, that this may change with future releases, especially with the new workflow manager that is in the works.

Each projekt is stored in its own directory in the workspace directory.
In that project root is a file "workflow.knime". This is an XML text file which contains the structure of the pipeline. Each node of the workflow is stored there (with its path, ID, name, user name, etc). And the connections between the nodes. Also, the values of the user settings appear there, but are kinda ignored.
For each node in the workflow a sub dir is created, named after the node with its ID.
This node dir contains two sub dirs: "data" and "internal". "data" stores the output data of the node: for each port exists a sub dir (data_0, data_1, etc.), holding the actual data of the table (data.zip), some meta information about the zip file (data.xml) and a file with the structure of the data table (spec.xml), like number of cols, col names, domains, etc.
The "internal" dir contains whatever the nodemodel writes out during save (in the saveInternals() method), for example the data model, or view content, etc. It is implementation dependent.
The node settings (parameters entered in the node's dialog) are stored in the "settings.xml" file, you'll find in the node's root (next to the "data" and "internal" dir). The content is different for each node (as the dialog and the parameter set are). It is always a key value pair though.

I don't know if this is helpful in any way, and it doesn't really qualify as easy to retrieve the information from that structure I guess. But it's all there. :)

Regards,
- Peter.

P.S.: Did I mention that we may change the way flows are stored in the future? Yes? Oh, good.
P.P.S: this thread could help retrieving some of the information: http://www.knime.org/node/218#comment-334

#7