Slurm & HPC

Is there any literature or proof of concept around integrating a Knime Server into a scheduler such as Slurm? I believe there was/is integration into the Sun Grid Engine?

Thank You

1 Like

Hi @awhelan,

With a small tweak, a KNIME Executor located on a shared drive can be launched multiple times as a SLURM job in a cluster. This is inline with the the distributed executors architecture of KNIME Server which would allow you to use multiple nodes of an HPC as KNIME Executors.

We are working on a blog post describing this exact scenario. It should come out soon. I will be more than happy to give you an update when it gets published.



This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.