Cmd program with ProcessBuilder in KNIME node doesn't work on Windows?

[Blarc]

Hi,

I am developing a KNIME node for CMDock. The node just runs the CMDock program as subprocess with ProcessBuilder class. It works on Linux, but not on Windows.

The main code that is responsible for running the program can be seen here.

I also wrote a standalone java program, that just runs the program with ProcessBuilder and it works fine. I also noticed, that the program runs fine, if I run KNIME with eclipse debugger.

This makes me believe, that KNIME is probably missing some permissions on Windows for running the program.

Any help is very welcome,

Jakob Maležič

---

Code for standalone java program that works:

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import java.util.function.Consumer;

class ProcessBuilderTest {
	public static void main(String[] args) throws InterruptedException, ExecutionException, TimeoutException {
		ProcessBuilder builder = new ProcessBuilder("cmd.exe", "/C", "C:\\Users\\jakob\\Downloads\\cmdock-v0.2.0-win64\\bin\\cmdock.exe", "-h");

		int exitCode = 0;
		try {
			Process process = builder.start();
			StreamGobbler streamGobbler = new StreamGobbler(
					process.getInputStream(),
					process.getErrorStream(),
					System.out::println,
					System.out::println
			);
			Future<?> future = Executors.newSingleThreadExecutor()
					.submit(streamGobbler);
			process.waitFor(600, TimeUnit.SECONDS);
			future.get(600, TimeUnit.SECONDS);
			System.out.println("Finished.");
		} catch (IOException exception) {
			System.out.println(exception.getMessage());
		}
	}

	private static class StreamGobbler implements Runnable {	
		private InputStream inputStream;
		private InputStream errorStream;
		private Consumer<String> inputConsumer;
		private Consumer<String> errorConsumer;

		public StreamGobbler(
				InputStream inputStream,
				InputStream errorStream,
				Consumer<String> inputConsumer,
				Consumer<String> errorConsumer) {
			this.inputStream = inputStream;
			this.errorStream = errorStream;
			this.inputConsumer = inputConsumer;
			this.errorConsumer = errorConsumer;
		}

		@Override
		public void run() {
			new BufferedReader(new InputStreamReader(inputStream)).lines().forEach(inputConsumer);
			new BufferedReader(new InputStreamReader(errorStream)).lines().forEach(errorConsumer);
		}
	}
}

[sonnenburg]

Dear Jakob,

I’m running external processes like this, also using ProcessBuilder, but simply redirecting output into files:

    ProcessBuilder processBuilder = new ProcessBuilder(args)
            .redirectOutput(Redirect.appendTo(m_logPath.toFile()))
            .redirectErrorStream(true);

Some remarks to your code:

• I don’t use “cmd.exe /c my_exe arg1 arg2 …”, instead directly: “my_exe arg1 arg2 …”
  But in your referenced knime-extension code, you don’t do this neither, so no difference here.
• You are using the wrong path separators “:”, for windows it has to be “;”, in your extension code:
  env.put("PATH", String.format("%s\\bin:%s", ...));
• Library (dll) path specifications need to go also to the PATH environment variable on windows, there is no LD_LIBRARY_PATH on windows.

Best, Frank

[gab1one]

Hi @Blarc,

Great to hear that you are working on a new KNIME extension. Can you explain exactly what kind of error you are getting? Take a look at the KNIME log (View → Open KNIME Log) after enabling DEBUG logs.

Did you try attaching a remote debugger to the AP on Windows, to investigate the issue?

best,
Gabriel

[Blarc]

Dear Frank,

first of all, thank you so much for helping me. I believe I managed to fix it. One of the errors was definitely the wrong separator. But that got me thinking if there is something else wrong with environment variables, so I printed them out and quickly saw, that Windows uses “Path” instead of “PATH”.

The node is now working as expected on my computer. I do find it a bit odd that I had to change “PATH” to “Path”, since CMDock’s scripts also set “PATH” (source). Will need to test a bit more to be really sure everything works. Maybe set both “Path” and “PATH” just to be sure.

Thank you,
Jakob

[sonnenburg]

Hi Jakob, good to hear that your problems are solved now!

Concerning Path / PATH on windows: Yes, you are right, if you execute “set” in a cmd terminal, the path variable will be given as “Path”, but generally environment variables on windows are not case sensitive, it should make no difference - but I have to admit I didn’t try this with java. I tried this with python and cmd. Here is a tiny example with an echo bat-script in the “subdir” sub directory:

D:\tmp\230516_win_path>subdir\hello.bat
hello from subdir

D:\tmp\230516_win_path>hello.bat
Der Befehl “hello.bat” ist entweder falsch geschrieben oder
konnte nicht gefunden werden.

(= command not found)

D:\tmp\230516_win_path>set path=%path%;D:\tmp\230516_win_path\subdir

D:\tmp\230516_win_path>hello.bat
hello from subdir

Best, Frank