Finding (recent but not that recent!) "recent workflows"

[takbb]

I’ll admit that I’m not always the best at organising my workflow names and just occasionally I want to go back to a workflow I was working on last week (or yesterday!) but cannot remember exactly what I called it.

Ideally with KNIME, I’d like a version of the KNIME Explorer that can be displayed as a grid rather than a tree view, and is then sortable by modification date, and maybe includes a column containing the initial part of the workflow “description”. This would certainly help me in these situations.

I know there is a “recent workflows” menu, but quite often (especially if I’ve been having a play, or helping out here on the forum), my workflows from last week have long since dropped off the bottom. I know also that I can increase the number of “recent” names it remembers, by setting this in preferences… but… anyway, let’s just say there will always be occasions for me when it is insufficient.

So I look up and down the KNIME explorer pane which isn’t giving me any clues and I click in workflow group folders to no avail… Then self-doubt kicks in and I begin to wonder if I even saved it, or maybe I just dreamt it !

If this sounds at all familiar then this component, that I’ve placed on the hub, may be of assistance to you too.

Recent Workflows – takbb

Drop it on a blank workflow canvas (or even an existing one) and execute the component.

It might take a few moments (and sometimes quite a few moments) to read the local workflows folders.

When its status goes green, open its output data port. It will list all of your workflows in descending order of last update.

It would be nice if it were actually possible to open a workflow from this component, but I’m not a miracle worker! Simply discovering what the workflow is called (and if it is contained in a group folder) should be enough for me to then manually locate it in the KNIME explorer.

Anyway, if this output looks at all familiar it might be because this was originally written as a demo workflow in July 2021. But having had need of it recently, I decided to make it into a component… to make it easier to find… because guess what… I couldn’t remember what I’d called it…

[Martyna]

Hi @takbb ,

thanks a lot for sharing this with the community and on the hub, sounds very useful!
I have days where I produce a lot of different workflows to test something, Will for sure test your component

Thanks!
Martyna

[takbb]

Thanks @Martyna , I hope it is useful.

[badger101]

Hi @takbb , first of all, thank you for taking the time to invent new things and share it with others

Is it possible that you can add an option for the user to dig deeper, such as showing the list of nodes/components used in each workflow. I imagine this to be as simple as adding another column of list type to the existing table you showed in the screenshot above.

The background of my case:

I have used an uncommon node maybe 1-2 weeks ago, but I forgot which workflow I had it on. (I don’t even remember the exact name of that node but I will if I see it on a list.) If I can revisit the workflow where the node was on, I don’t have to create a new workflow with all the fuss of uploading/re-uploading raw data and redesigning from scratch.

Hope that makes sense?

[takbb]

Thanks for the kind words @badger101 …

I can see this could be useful. As far as I can tell, finding the nodes is a case of iterating the subfolders for each workflow. This could take a bit of time especially with a large number of workflows, but taking your idea, here is a component that you can attach to the Recent Workflows component, and it would do something like what you are asking. I don’t know if assumptions I’ve made about the naming of nodes will always hold true, but give it a go and see if it works

Recent Workflows Node List – takbb

There is no config here, and it takes as an input table the output of the Recent Workflows node. You could of course put a row filter, or similar, in between if you want to limit the workflows that it will process.

[badger101]

Thanks @takbb , I just logged in, and will have a look at that and provide updates

[badger101]

Update:

Thank you so much @takbb . I’ve tested it on my dummy workflow and I like that it iterates the nodes inside components and metanodes as well:

I also noticed that workflows that are recently deleted prior to running this component are not included.

Really appreciate it!

[takbb]

I thought I’d revisit an old feedback topic.

In the more recent versions of KNIME, when using Modern UI, there is a “recent workflows” page which gives some assistance in find a workflow you were recently working on. Classic UI has “recent workflows”

Sometimes though, I still want to be able to search for an old workflow name and I cannot recall how long ago I last worked on it

So in spite of updates to KNIME, this has still led me back to the component I wrote back in 2022 (above). The component achieved the objective, but it one big failing was that it was slow to find the workflows.

The reason for its pitiful performance was ultimately down to my choice to simply ask List Files/Folders to traverse every subfolder in the current workspace, looking for workflow.knime files. That can be a huge number of subfolders for a very small number of workflows. It works, but it is an incredibly inefficient approach.

So I had a rethink, and decided there were some optimisations to be made. With the introduction of a recursive loop and judicious use of List Files/Folders within the loop (which now doesn’t search any subfolder of a folder than contains a workflow.knime file), rather than have List Files/Folders blindly search every single subfolder in the workspace, design/dev effort increases a little but run-time effort is reduced and the performance skyrockets!

When I said design/dev effort increases “a little”, well maybe it’s a little more than a little:

V1

becomes…

V2

Replacing that single List Files/Folders node with the above nested loop might be counter-intuitive, but it now finds all the workflows in my workspace in about 0.5 seconds, compared with the 90+ seconds it was previously taking.

So, if you can’t recall what workflow you were working on a couple of weeks ago, or which subfolder it was in, you can give this component a try, and no longer have to wait for the results

I’ve written it with KNIME 4.7.8 (the oldest version I still use) to ensure compatibility across the most recent versions of KNIME.

Recent Workflows – takbb

My reason for posting this, aside from hoping that others may find it useful, is to demonstrate that there can be many ways to achieve an objective in KNIME, but also it shows that rethinking the mechanics of an algorithm, and investing a little more effort in identifying and re-working the inefficiencies can sometimes give a dramatic pay-off.