I understand that Job Pools are used to keep workflows in memory for faster execution. We have 2 workflows which we are consuming using their REST endpoints. One takes input from user and the other does not take input from user. We are running in distributed mode and have 2 executors.
The results are as follows ::
There is no significant performance gain when job pools are enabled.
The job pools are set by following this documentation :: https://docs.knime.com/2018-12/server_admin_guide/index.html#knime-server-job-pool.
Please provide insight as to why job pools are not reducing the average execution time.
I guess that there are a couple of possibilities. One is that the ‘load time’ of the job is negligible compared to the execution time. Could you document how you are calling the pooled and non-pooled jobs via the REST API?
I am using Apache Jmeter which spawns multiple threads and hits the specified REST end point and calculates the timings. I am currently setting 10 threads to hit my REST endpoint using both the job pool (:job-pool) and non job pool options (:execution).
Thanks for the summary Sheldon. It’s possible to use POST to the :jobs endpoint which will load a job. From that it is possible to benchmark the load performance of the workflow. In the case that is a long time, then I would expect a large difference between :pool and :execution endpoints.