Hi,
Is it possible to use job pooling when you schedule a workflow?
I set the properties of the workflow as below:
And I scheduled this workflow to run regularly for few hours. Will the job pooling works with these settings?
Regards,
Nithin
Hi Nithin,
There is only the option to either execute job pooling or job scheduling. It is not possible to combine both.
Job Pooling was designed to to save loading time of jobs. In case your job execution time is smaller than the job loading time, this concept would be applicable.
Both options are also available via REST API
Job Pooling
http://<server-address>/knime/rest/v4/repository/<workflow>:properties?com.knime.enterprise.server.jobpool.size=<pool size>
Job Scheduling
http://<server-address>/knime/rest/v4/repository/<path>:scheduled-jobs
You can find more information about the KNIME Server REST API in our Swagger Documentation:
http://<server-address>:8080/knime/rest/doc/index.html
Best regards,
Kevin
2 Likes
Hi Kevin,
Thank you for your reply. I am scheduling jobs in server via analytic platform:
Right click on the workflow in server → Execute…
I have to run the same workflow 4 times at the same time every minute for one or two hours. From previous executions, it is clear that the loading of the jobs itself is taking between 4 to 10 seconds, depending on how many parallel execution I have scheduled. So I want to use job pooling to reduce the load time. Is there any way to achieve this?
Another question related to job-pooling:
From sever documentation it is clear that the job pooling is possible for one step exxecution (:execution end point). In my project I have to make the rest call asynchronous, i.e. multi step execution via :job etc.
Is there any future plans (or available now) to make job pooling available for multi step execution?
Regards,
Nithin
Hi Nithin,
Regarding your request about job pooling for your scheduled jobs, the answer needs to remain the same. You are only able to allow job pooling or job scheduling, there is never both.
Regarding the multi-step-execution, do you mean something like a WebPortal execution? In such a case, it wont work with job pooling either, since job pooling is really only meant for (“one-shot”) executions e.g. via REST API.
Best regards,
Kevin
Hi Kevin,
multi-step-execution: not via WebPortal but more or less same via rest api. Create a job for a workflow via :job end point and then execute the job via another rest call using the job id. But I asssume the answer is same. Correct me if I am wrong. Is there any plan for future develoment in this area such as job pooling for executions other than one-shot executions?
Regards,
Nithin
Hi Nithin,
I’m sorry for the delay in response. You are right, the answer in this case is still the same, there is either job pooling or job scheduling.
Due to the different nature of both features we will keep them as separate features until now. We surely will work on future development on these options. However, I cannot tell how this will look like precisely in future.
Best regards,
Kevin
1 Like