Clever way to 'reuse' a workflow?

Hi All,

I frequently run into a situation where i have a workflow that i routinely use in a standalone way, feeding starting data in via a Table Creator node, but which does something that would be useful when built into a larger workflow.

Is there a clever way to do this in Knime?

All i can think of currently is maintaining two copies of the workflow, one for standalone use which is self contained, and another with inputs and outputs to/from the workflow which can be used by other workflows.

All the best,

Mirri

1 Like

Hi @Mirri , one way to do this is to convert your reusable workflow into a component, which you can then use in various workflows. And if you keep the component linked, it will update automatically in the workflows whenever you modify the component, instead of having to go in each workflow that is using it to manually update it.

If you come from a programming background, you can think of a component as a function that’s standalone and can be resused.

I built a lot of “libraries” as components that I keep reusing in various workflows.

5 Likes

Hello @Mirri,

seems to me you are looking for container nodes. For example use Container Input (Table) node after Table Creator node. Then if you call your workflow with Call Workflow (Table Based) node you will start your workflow with data from another workflow (calling workflow) while you can still run your workflow as standalone use with data from Table Creator.

Br,
Ivan

6 Likes

Thought the same as @ipazin
@bruno29a Can you elaborate on some examples of your reusable components? Great idea

1 Like

Hi @Daniel_Weikert , sure I can elaborate on some examples of my reusable components.

First, just to comment about the Call Workflow + Container Input, it’s also a good alternative. It will depend on what you want to do as always. I used to use these 2 when I first started using Knime. Coming from a solid programming background, I was building my workflow in multiple smaller workflows where I separated my processess in standalone workflows, a bit like library files that you can “import/include” via the Call Workflow, until I found out about components, and migrated this process to components.

Now, I’m not saying that we should use components instead of Call Workflow+Container Input. It comes down to what you want to do. In my case, components make sense. In other cases, where you got various whole workflows to connect together, then you may want to use Call Workflow+ Container Input.

Now, to elaborate on some examples of reusable components, I have quite a few. The way I set it up is I have a folder called Shared Library, and it’s where I store my libraries of reusable components.

Drilling into the folder, I have 2 subfolders, Source and Runtime. The Source folder contains all the source of my components. It’s where I work on the components, build them and modify them. And the components are categorized in different folders.

The Runtime folder contains the components. It’s from where I reuse the components. Of course, whatever is in the Runtime folder are also in the Source folder:
image

For the Demo, we’ll look at one of my Misc components, which is the “Move first row as Headers”. The source workflow looks like this:

So, I built the workflow to move the first row of the table as headers, and then converted the process as a Component. Next, I create the Component as a standalone node into my Runtime folder like this (Right click → Component → Share):

You can even see in the tooltip that it says “Saves the component as a reusable shared component”

And I “Share” it into my Runtime/Misc folder:

For the Demo, I built something like this as my starting point:
image

And data looks like this at this point:

I want to move the first row as headers:

I just need to drag my “Move first row as headers” component to the workflow, just like you would drag a Knime node from the Node Repository to the workflow, and now the workflow looks like this:
image

And results as follows:

Now, you see the little green arrow in the component?
image

That means that it’s linked to wherever the component came from, in my case it’s from Runtime/Misc/Move first row as headers

What this does is that, if I ever modify Runtime/Misc/Move first row as headers, this node will be updated automatically with the changes. So, imagine if you have multiple workflows that are using this component. You just need to modify the source component (source here is where the component spawned from), and all of the workflows that are using it will be updated automatically.

We can add other components (just drag and drop the Runtime version):
image

Here’s the results of my “Get RowCount as variable” component:
image

For updating the components, basically, you get a popup window like this when you open your workflow:
When I had only 1 component:
image

After I added the second component:
image

That’s where it will check for any updates to the components and update the components accordingly.

You can get a copy of my “Copy or Move column headers to first row” component here

Similarly, for @Mirri 's case, you could have the sample data generation as a component.

Let me know if you have any questions.

7 Likes

Thank you ipazin, this is exactly what I was looking for!

Many thanks,

Mirri

3 Likes

Thanks a lot for the detailed answer. I was in particular curious about the workflows itself. I have used components a few times in the past but neglected to create them for reusability

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.