Hi -
Can anyone guide me how to extract the list of workflows along with node details and the configuration details (metadata) for all the nodes.
Currently fetched the workflows along with node details but stuck in extracting node configuration details for all the workflows.
Hi @narayanan ,
I’m sorry for the late reply: Assuming you are building a workflow that does all that, and you have the workflows already by ID or Path, you can use a Workflow Reader to read them in and then a Workflow Summary Extractor to get all information about the workflow, including the configuration of each node. Would that help you already?
The caveat is that all installed extensions need to be present wherever the workflow runs to get the workflow summary.
Would you share what your ultimate goal is? I’m asking because we are currently working on a (set of) workflows that do checks, e.g. if a freshly uploaded workflow has some undesired settings like stored credentials. I’d be happy to share a first version you can build upon and help you get along faster and also to get some early feedback on our side. Let me know, and I’ll share it with you with some instructions!
Best regards,
Lukas
Hi Lukas,
Thank you for your message and update on the topic below. I would greatly appreciate your guidance in resolving this issue effectively.
As KNIME Business Hub Global Admins, we need to identify which teams are using SAP Connectivity (DB Connector & Post Request) in their workflows. To achieve this, we aim to filter the nodes related to SAP connectivity and extract the parameters configured on these nodes across all workflows in the KNIME Business Hub environment.
Currently, we have prepared a workflow to address this, but it is taking a significant amount of time and not producing the expected output.
Specifically, we noticed that while using the Workflow Reader and Summary Extractor nodes, it seems the workflow is being executed during the extraction process and taking very long time.
Could you please provide your inputs and suggestions for optimizing this process? Additionally, it would be helpful if you could share a sample workflow or best practices for reference.
Looking forward to your valuable insights.
Please find the attached workflow created using Workflow reader and workflow summary extractor
DB Workflows.knwf (112.4 KB)
Hi @narayanan,
thanks for the workflow! I suggest there are two parts to it:
For 1.: That can get quite resource-intensive, as yes, each workflow needs to be loaded to get the node configurations (which does not execute, but might take a while, depending on the size of the workflow to be loaded). You’d probably also need to consider workflow (and component) versions – people might have removed a connection. Luckily, we have some components that can assist there. On the positive side: We can get filter all workflows by the relevant nodes first, before doing the expensive workflow loading. I uploaded an example that does all that here:
From there, you’d need to evaluate the node settings that are relevant for you and then decide on an action. I’d also recommend to write out partial successes in the loop, e.g. by appending to a excel file, in case something goes wrong at some point.
For a solution for 2. we are currently working on, I’m contacting you via Email.
I hope that brings you forward already and may serve as an inspiration!
Kind regards,
Lukas
Thank you Lukas for the reply and checking, workflow. Much appreciated
Really that’s good point of using specific extension filtering out initially instead of checking all the workflows.
One small query, the output of the above workflow doesnt show workflow name, node and parameter configuration. the output coming as json path with all the information.
Can you please guide me if i am missing here anything.
Hi @narayanan ,
I’m glad it’s useful!
You can e.g. use a Variable to Table Column node to add back the workflow identifier and use e.g. an Expression node to extract the workflow name from the path.
The node configuration you’ll find in the settings field in the workflow summary. They contain a key, which is the identifier of a specific setting.
I added these functionalities to the workflow here:
Let me know if you need anything else!
Kind regards,
Lukas
