Calling one workflow from multiple call local workflow

#1

Hi, I am new to Knime Server and still trying something for future development.

I have a basic question.

For example, I have a single workflow that does classification scoring and evaluation of a ML model (and some custom scripts too).

I have 5 workflows running different use cases, but the 5 are classification tasks maintained by 5 users.

Considering I ran these 5 workflow in parallel, will the 5 be able to consume the single scoring workflow from the call local workflow node at the same time?

So in this case, we might run the scoring workflow at the same time by 5 processes.

Is it doable in Knime server? We don’t want it to be sequential like waiting scoring for 1 model before it can score the other model.

A different case is: maybe can a single workflow called multiple time while it is still running? Considering a workflow takes some time to run, but it is used by different user groups with different “filtering” purposes. They maybe want to run the same workflow at the same time, but unbeknownst to each other.

Looking forward for some input.
Thanks!

0 Likes

#2

Hi Mizunashi92,

Workflows on the KNIME Server will run in parallel subject to the executor resource constraints. Therefore if you have enough compute resources to run the 5 workflows in parallel, quickly enough on the hardware that you have available you should be fine.

As an aside, you may want to look into the newer Call Workflow (Table/Row) based nodes. They can automatically detect whether they are running locally or on the server and act appropriately. This has a benefit if you are using the distributed executors functionality to horizontally scale execution of workflows.

Regarding your last question about running the same workflow multiple times in parallel. That is also possible. Practically the KNIME Server makes a copy of the workflow before executing a job, so in your case two independent jobs would be running at the same time. Some limited consideration may be needed if you read/write to a shared resource (e.g. file) from each job. However, that can be easily solved by using the ‘Create Temp Directory’ node.

Best,

Jon

2 Likes

#3

Thank you very much,

This alleviates my concerns.

Thanks!
Mizu-

0 Likes

closed #4

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.

0 Likes