Increase max number of working threads above 100 for 240 threads machine?

nba · October 14, 2019, 3:58pm

Hello,

I would like to know how to increase the max number of working threads above 100.

Is it possible to use special configuration settings in the .ini file or anything else?

Thanks

Vernalis · October 15, 2019, 9:16am

You might be able to set it in the knime.ini file. Based on:

github.com

knime/knime-core/blob/da17be0c6253dba44246be477f439b4525423941/org.knime.core/src/eclipse/org/knime/core/node/KNIMEConstants.java#L488-L505


int maxThreads = Runtime.getRuntime().availableProcessors() + 2;
String maxThreadsString =
    System.getProperty(PROPERTY_MAX_THREAD_COUNT);
try {
    if (maxThreadsString != null && maxThreadsString.length() > 0) {
        int val = Integer.parseInt(maxThreadsString);
        if (val <= 0) {
            throw new NumberFormatException("Not positive");
        }
        maxThreads = val;
    }
} catch (NumberFormatException nfe) {
    // no NodeLogger available yet!
    System.err.println("Unable to parse system property "
            + "\"org.knime.core.maxThreads\" (\"" + maxThreadsString
            + "\") as number: " + nfe.getMessage());
}
GLOBAL_THREAD_POOL = new ThreadPool(maxThreads);

and

github.com

knime/knime-core/blob/da17be0c6253dba44246be477f439b4525423941/org.knime.core/src/eclipse/org/knime/core/node/KNIMEConstants.java#L108-L111


/**
 * Java property name to specify the default max thread count variable (can be set via preference page).
 */
public static final String PROPERTY_MAX_THREAD_COUNT = "org.knime.core.maxThreads";

using -Dorg.knime.core.maxThreads=200 may work - although I don’t know if this will be ignored somewhere else within KNIME.

(Incidentally, each node generally will only use the maximum of either this value or 1.5 × the number of CPUs, unless you have written your own nodes which manually provide different values)

Steve

nba · October 15, 2019, 9:22am

Hi Steve,
Many thanks for your reply, appreciated

nba · October 23, 2019, 9:50am

Hi Steve,

Just to keep you updated, on my machine with 120 threads, Knime automatically chooses 120 as the default max number of threads in Preferences. Thus it seems the 100 limit depends on the machine.

Nicolas

system · June 23, 2023, 8:37pm