Can you develop an extension that doesn't use nodes?

Essentially I am trying to make a specific kind of logging for KNIME. I want this to be an easy plug and play extension for KNIME rather than doing something with the source code, is this possible? I also want to know if the properties of existing nodes and custom nodes are exposed for this purpose. Other things I might want to be able to access include the users actions and the existing KNIME log. It would also be great to get pointed in the right direction for resources for developing for KNIME from this perspective, rather than creating more nodes.

Thanks!

Hello @azsb1g19,

and welcome to KNIME Community!

What are you missing from KNIME logging? For example KNIME Streaming Execution (Beta) extension comes without nodes so it’s possible to create such an extension. However I’m not a developer so can’t direct you more that this.

Guess you already checked the Quickstart guide from KNIME Docs?
https://docs.knime.com/latest/analytics_platform_new_node_quickstart_guide/index.html#_introduction

Br,
Ivan

2 Likes

KNIME logging is great! I’m making logging that can feed into an existing body of work that stores that logging information as a different kind of graph that is queryable by an SQL-Like language. That streaming extension looks interesting though I’ll take a look, and the quickstart guide I haven’t gotten around to yet but thanks all the same,

2 Likes

Hello @azsb1g19,

ok. So you are in for log file analysis using SQL-like language. Interesting. However maybe it’s not necessary for you to develop your own logging but rather transform current log entries to what your “existing body of work” requires? Additionally here is workflow from Hub that does (Server) log file parsing&analysis.

Anyways good luck and if any questions/issues feel free to ask.

Br,
Ivan

2 Likes

Hi @azsb1g19

If you want to write the logs to a new location you can very simply add a custom slf4j log binding:

best,
Gabriel

3 Likes

Hi again, I have a few more questions.
Is it possible to look at the source code of the extension linked? I’m unclear on how to practically develop this idea further. In particular, the first step of the quickstart guide seems specific to developing node extensions. I’m probably meant grab something different here than the KNIME Node Wizard right?


Any further details would be much appreciated.

Hi @azsb1g19

If you don’t want to develop a node extension, then you don’t need that wizard, in that case you can just the eclipse plug-in wizards to develop a regular osgi eclipse plug-in that provides the functionality.

best,
Gabriel

1 Like

Thanks for your help so far, I have a few more questions:

Conceptually, for every node in a workflow I would like to collect a list of all of the data items input to the node and a list of all of the data items output from the node. I would also like to read the annotation for the node. In practice I think I should manipulate the FlowObjectStack in the NodeModel class but I’m not sure how. I’m confused by the stack including input and output variables. I’m also not sure how to get the annotation. I don’t necessarily need this information during runtime, but I would need to be able to infer this from logs I generate.

I haven’t worked with OSGi yet but I’ll go through quickstart resources online. I’m unclear about how to “snoop” and look at the data I need during runtime. If there’s a technical term for this that I can go away and look up that would be appreciated. For my situation I’m unsure how to make my code do something only when data is processed through nodes, and collect data when that happens. Sorry if that’s a bit vague.

Thanks again

Hi @azsb1g19

Have you looked at the Workflow Summary Extractor – KNIME Hub node? It outputs a lot of metadata about a workflow, including annotations and connection information.
You can then feed it into a component like this one: Analyze Workflow Summary – KNIME Hub

You can also create workflow summaries from executed jobs on the KNIME server using a REST call, see this section from the REST API documentation:

These are aimed at workflow audit tasks.

They make use of this class: knime-core/WorkflowSummaryCreator.java at master · knime/knime-core · GitHub to navigate the workflow and extract the information.

best,
Gabriel