Start a Workflow, e. g. a database query, after a certain time

Loop end would collect the data that comes its way. Variable Loop End would just end the loop and would collect Flow Variables (so you could continue some other tasks afterwards or hand down information from within the loop) - the data handling would have to be done separately within the loop (save to database or disk).

In your screenshot the loop end is within the component. Question is if you want that you would want the component to finish. So it might be better to place a Loop End outside.

@mlauber71

ok understand.
If I patch the node inside the component, it “only” terminates the component but not the workflow.

But the component is also part of the workflow - where do I set the loop end node correctly then?

Loop wo platzieren

:thinking:

greetings

@USCHUKN1ME you will have to have everything that should happen within one iteration of a Loop within the loop. If you have a more complicated setting that might involve some trial and error. And you will have to make sure to ‘reset’ scything you want to be new/resetted at the beginning of each iteration. If you want to transfer information from one loop iteration to the next there are the Recursive Loops (which I still somewhat struggle to grasp :slight_smile: ).

If you want to store intermediate information you can -well- save that as a table and read it back in the next iteration (not the most elegant of solutions).

Then I would suggest you familiarise yourself with the concepts of Loops and Flow variables in general

@mlauber71 thank you, i guess it isn´t easy :see_no_evil:

PS: Many of the links doesn´t work!

@mlauber71 hello again

The problem for a beginner, with such large workflows, is to understand why Node A,B,C,… are required for the loop nodes or perform other functions.

Loop Start
Example Loop by mlauber-LOOP-Start

Loop End

How to arrive at this configuration is not (yet) revealed to me!

I always think of a loop node an if… then… else or a for… to loop.

But I think that’s wrong!

greetings

If then else is another matter and there are other nodes for that.

With loops there are tow important End nodes.

  • one would collect i a KNIME table whatever output is there (you can allow changing structures)
  • the variable Loop end would just end whatever is within the loop and just store flow variables with possible informations in them

My impresssion is that you might want to take a step back to the drawing board and make a plan what you want to achieve and familiarize yourself with some KNIME nodes and concepts mentioned in this thread.

Regarding automation / scheduling you have these options

  • KNIME server (possibly via AWS)
  • Batch mode via CRON or Task Manager
  • a wait construction I showed above

@mlauber71

actually, it’s relatively simple.
I have a database query and a workflow behind it (end with output of a graphic/dashboard + sending an email). I want to restart the database query + workflow at three predefined times without having to manually reset and rerun the individual nodes.

Start:
Datenbankworkflow Anfang

Workflow after DB Query Node

End of the workflow in the component

My Question?

Which loop node is the right one for this ?
Is it the Wait Node with a Table creator Node before?
:thinking:

Sorry - which one do you mean?

CRASHED :expressionless:

Next try :stuck_out_tongue_winking_eye:, again via the WAIT node. Have now built something about
the task scheduling of Windows, which Starts a batch file which creates
a simple text file and overwrites each time. I configured the WAIT node
in KNIME to wait for the modification of a file and then execute it. That worked.

I try to start the subsequent wokflow behind the WAIT node via a LOOP node. :thinking:

Stupid about the WAIT node is, that it is only executable once and cannot be controlled with it.

Or is there any way to control the WAIT-Node?

WAIT-Node

If anyone here has a bright idea I am very grateful for a tip.

greetings

My suggestion would still be to try and put the wait node within the loop and create the times of a day you want it to be executed in a table to control when the loop would continue.

You might want to download the example and maybe try to adapt it so we could try and discuss what you would need.

You could of course also use a task manager to start a batch file or even start it at several times a day which in any case would need a computer to run continuously which also would be true if you just let the knime workflow wait with the loop.

If you use the batch mode it might be necessary to use a different knime-workspace if you want to use the machine at the same time.

The knime server in a dedicated server would be another option of course.

First @mlauber71: Thank you !

My first question about your case workflow would be:
What is the function of add_seconds?

I’m trying to figure out all the functions of your workflow…

Greetings

@USCHUKN1ME I tried to build a system you might be able to expand and use.

The idea is to have a workflow that would iterate a number of times (v_number_of_intervals) with a set interval (of seconds) between (v_seconds_interval) them. The workflow looks for a trigger_file.csv if this file has changed from the last time the loop has run a component would be executed. If the file has not changed nothing would happen.

Admittedly not the most elegant way but you could build a small automatic execution machine with this as long as your workflow does run.

You can monitor the effect. Every time the trigger file changes the component gets executed. Here there are 10 iterations. At the second iteration the component executes, then nothing happens and then at the iterations 7, 8 and 9 the file has changed and the component executes again (the changes are simulated in the m_001 workflow - in a real live scenario they would occur by outside factors).

This might serve as an example how such a thing could work. Obviously it would need some work and one can debate if this is the best way to go forward. I would recommend using the KNIME server instead :slight_smile:

(please download the complete workflow group)

This job m_001 is there to prepare the setting and simulate the changing tigger_file.csv

You have three components:

  • trigger_file.csv - a simple CSV file that would change over time (a new delivery might be send to a folder)
  • trigger_meta_list - a small database that would store all the scans of the trigger_file.csv so you could see if something has changed
  • sample_00 - a dummy database in the same H2 database where you would either do something (if the trigger file is newer= or not

image

:grinning: :+1: wow - thanks a lot @mlauber71 .
I´ll trying to follow your workflow.

About KNIME-Server:
Kosten für einen KNIME-Server :thinking:
In my opinion, an Investment of 12,500 euros for a workflow is
somewhat disproportionate.

Greetings

1 Like

@USCHUKN1ME this very much depends on your business model and what you want to do with it. The KNIME server is much more than one workflow :slight_smile: - I have a collection about what is possible there:

Also you can use KNIME server with AWS and just pay what you use :slight_smile:
https://www.knime.com/knime-software-on-amazon-web-services

And maybe if you contact the KNIME team they might help you with a test license so you can see for yourself …

1 Like

I know I’m very late to the party and maybe given all the replies after this question my reply might not be needed anymore. Still I think it fits to this question. About “times loops” please look at the vernalis community extensions. Yes, they are in the cheminformatics/life science space but also have some general use nodes like timed loops:

image

Maybe these can be helpful for your workflow or workflows in the future.

3 Likes

Hello @kienerj ,

thank you.

You are not too late :slightly_smiling_face:

Chunk Loop Run-to-time Loop Start(A)

Chunk Loop Run-to-time Loop Start(B)

Those are two interesting loop nodes. Which TN take a closer look at.

First, though, l’ll take a closer look at @mlauber71 Suggestion, as it’s
interesting as well.

Greetings

1 Like

You could absolutely combine the approaches, I would think.

Thank you @kienerj for pointing to this feature.

Another interesting function could be this component by @gab1one that would wait for a file to change. In order to bring this into a ‘looping system’ - some changes might be necessary. In my example I tried to do a small such system with generic KNIME nodes.

@kienerj which Loop-End-Node is the right one for this two Loops?

Have to figure that out yourself. Only really used one of the timed-nodes once. My comment bascially was: I know these exist, have a look maybe they help with your problem.

1 Like