Parallel execution with doParallel on KNIME Client vs Server

KNIME Server
1

I have problems with parallel execution using the doParallel package on KNIME Server (v 4.4.1). The funny thing is, it works fine within the KNIME Client (v 3.3.2).

Minimal example: I have a single "R Source (Table)" node that contains only a minimal script registering cores:

library(doParallel)
registerDoParallel(cores=3)
knime.out <- data.frame(
  N_WORKERS = getDoParWorkers()
)

Workflow, see attachment.

  • When I run this workflow locally in my KNIME Client, it works fine. Finishes in seconds.
  • When I run just the R code on the Linux server (Red Hat Enterprise Linux 7.3) that hosts the KNIME Server and KNIME Executor, it works fine. Finishes immediately.
  • BUT: When I run the workflow on the KNIME Server, it keeps on running and does not finish.

Does anybody know why this is happening and what to do about it?

Cheers

Ben

2

Oh, I found the solution. I made a mistake, apparently registering the cores was not the problem.

Apparently, you MUST put the registerDoParallel(cores=N) statement into the same R node as the foreach() statement. Putting it into an "R Source (Workspace)" node and connecting that to another R node via the grey R workspace connection will make R forget about the registered cores and you'll use only a single core.