Failure to Run Two KNIME Batch Processes in Parallel on Windows

[Tokbanov]

1. Problem Description

I am unable to run two KNIME Analytics Platform (5.5.1) batch processes simultaneously on Windows Server. Individually, each process runs successfully. However, when launched concurrently, both processes fail immediately or shortly after starting.

2. Symptoms & Error Messages

The failure manifests in several ways:

• Primary Error: Process terminates silently after ~5 seconds with a native code crash in libcef.dll: EXCEPTION_ACCESS_VIOLATION (0xc0000005) and the message "The crash happened outside the Java Virtual Machine in native code" and two files are created hs_err_pid(some process id).mdmp and hs_err_pid.txt

• Configuration Error: Constant ConfigurationAreaChecker errors in the GUI regarding the lock file (C:\KNIME\configuration\org.knime.core\(username).lock), indicating access conflicts.

• No Logs: When the crash occurs, the .metadata folder and its logs are often not created in the designated -configuration directory.

3. Environment & Setup

• OS: Windows Server

• KNIME: 5.5.1, installed in C:\KNIME\

• Workspace: A shared workspace located at C:\ProgramData\knime-workspace.

• Launch Method: Custom batch scripts executed manually and via Task Scheduler.

• Isolation Attempt: Each process is launched with a unique -configuration parameter pointing to separate directories (e.g., C:\KNIME_CONFIG\process_1\) to avoid conflicts.

4. Steps Taken for Resolution

1. Isolation: Used separate -configuration paths for each process.

2. Chromium Disabled: Added -vmargs -Dknime.browser.activation=never to prevent crashes related to the Chromium engine.

3. Permissions: Verified full write/modify permissions for the user account on the entire KNIME and KNIME_CONFIG directory trees.

4. Logging: Enabled verbose logging with -consoleLog and various -D logging flags. The native crash often occurs before logs are written.

5. Clean Boot: Rebooted the VM to ensure no leftover processes or locks persist.

5. Current Status & Suspected Cause

The problem persists despite isolation and configuration changes. The evidence suggests a low-level, native code conflict rather than a high-level workflow or configuration issue. The most likely causes are:

• A race condition or conflict in a native library (e.g., libcef.dll, data processing libraries) when multiple instances are initialized simultaneously, even from different directories.

• The KNIME instance may still be accessing some shared resources in its original installation directory (C:\KNIME\), leading to lock file contention.

6. Request for Assistance

I need guidance on how to achieve true process isolation for parallel KNIME batch execution on Windows. Specifically:

• How to prevent the native code crash in parallel scenarios?

• Are there additional parameters to fully isolate KNIME instances?

• Could this be a known issue with a specific version or setup?

[mlauber71]

@Tokbanov I think in general KNIME batch processing will start its own KNIME version complete with knime-workspace. So, if you want to run anything in parallel you might need two separated KNIME instances complete with their own installation and workspace.

KNIME Batch Processing on Windows and MacOS