Cancel node with external process

Node Development
1

Hi all,

I have to start an external process in my node.  I managed this by (roughly)

Process process = new ProcessBuilder(getCallArgs()).start();
process.waitFor();

I'm using knime-3.4 with nodes implemented with api-2.

Now I want to handle cancel by user, such that the external process will be killed.

I found this post: https://www.knime.com/forum/knime-developers/kill-external-process-if-node-is-canceled with hints to ExtToolOutputNodeModel and CommandExecution, seems quite promising to cover my usecase.

Anyway, just being curious ...

When I start my node with the current ProcessBuilder implementation, I'm already able to cancel execution, which surprised me a little bit, because I don't do any cancel-handling in my code.  In the knime.log I found

:
2017-12-06 17:44:37,591 : DEBUG : main : CancelAction :  :  : Creating cancel job for 1 node(s)...
2017-12-06 17:44:37,591 : DEBUG : KNIME-Worker-2 : FlexX Docking : FlexX Docking : 0:1 : reset
2017-12-06 17:44:37,607 : WARN  : KNIME-Worker-2 : FlexX Docking : FlexX Docking : 0:1 : Execution canceled
2017-12-06 17:44:37,607 : DEBUG : KNIME-Worker-2 : WorkflowManager : FlexX Docking : 0:1 : FlexX Docking 0:1 doBeforePostExecution
2017-12-06 17:44:37,607 : DEBUG : KNIME-Worker-2 : NodeContainer : FlexX Docking : 0:1 : FlexX Docking 0:1 has new state: POSTEXECUTE
2017-12-06 17:44:37,607 : DEBUG : KNIME-Worker-2 : WorkflowManager : FlexX Docking : 0:1 : FlexX Docking 0:1 doAfterExecute - failure
2017-12-06 17:44:37,607 : DEBUG : KNIME-Worker-2 : FlexX Docking : FlexX Docking : 0:1 : reset
2017-12-06 17:44:37,654 : DEBUG : KNIME-Worker-2 : FlexX Docking : FlexX Docking : 0:1 : clean output ports.
2017-12-06 17:44:37,707 : DEBUG : KNIME-Worker-2 : WorkflowFileStoreHandlerRepository : FlexX Docking : 0:1 : Removing handler c8612e9b-f97b-4171-9bd2-13eec14e9732 (FlexX Docking 0:1: <no directory>) - 0 remaining
2017-12-06 17:44:37,760 : DEBUG : KNIME-Worker-2 : NodeContainer : FlexX Docking : 0:1 : FlexX Docking 0:1 has new state: IDLE
2017-12-06 17:44:37,818 : DEBUG : KNIME-Worker-2 : FlexX Docking : FlexX Docking : 0:1 : Configure succeeded. (FlexX Docking)
2017-12-06 17:44:37,833 : DEBUG : KNIME-Worker-2 : NodeContainer : FlexX Docking : 0:1 : FlexX Docking 0:1 has new state: CONFIGURED
2017-12-06 17:44:37,834 : DEBUG : KNIME-Worker-2 : NodeContainer : FlexX Docking : 0:1 : test 0 has new state: CONFIGURED
2017-12-06 17:44:37,836 : DEBUG : KNIME-WFM-Parent-Notifier : NodeContainer :  :  : ROOT  has new state: IDLE
2017-12-06 17:44:37,836 : DEBUG : KNIME-Worker-2 : ThreadPool :  :  : Future was canceled
:

The console log in the knime gui reports "Execution canceled" and I can reconfigure and restart the node.

But: the external process keeps running.  Now I'm wondering, what happens with the execute() method of my node?  I waited until the external process terminated (by checking the system's task manager), and apparently the execute() method won't continue after process.waitFor().  It seems that the whole NodeModel working thread had been killed.

Is using the ExtToolOutputNodeModel the only way to "catch" this case in the execute() method?

 

Thanks for all,

Frank

2

Wrapping the process.waitFor() with a try-catch reveils

2017-12-08 10:32:49,148 : ERROR : KNIME-Worker-7 : FlexxDockingNodeModel : FlexX Docking : 0:107 : java.lang.InterruptedException
    at java.lang.ProcessImpl.waitFor(ProcessImpl.java:451)
    at com.biosolveit.tool.Tool.execute(Tool.java:99)
    at com.biosolveit.flexx.docking.FlexxDockingNodeModel.execute(FlexxDockingNodeModel.java:259)
    at org.knime.core.node.NodeModel.executeModel(NodeModel.java:567)
    at org.knime.core.node.Node.invokeFullyNodeModelExecute(Node.java:1128)
    at org.knime.core.node.Node.execute(Node.java:915)
    :

and the InterruptedException is being caught in Node.execute(), which resets everything.

Hmm, maybe I can answer this by myself.  I assume, that doing a cancel will set the current thread's state to interrupted, right?

Normally, that has no direct impact, if I don't call ExecutionMonitor.checkCanceled().  Except when I use Process.waitFor(), which by itself checks the thread's interrupted state, and throws an InterruptedException if set.

So, when using Process.waitFor() I simply have to handle InterruptedException.  Or, use CommandExecution ...

The moral of the story, my java lesson for december: Always read the docs for which exceptions are possible, of course InterruptedException is mentioned in the waitFor() documentation!  :)

Best, Frank

3

I still was quite confused, because when I catch the InterruptedException in my NodeModel.execute() and throw a new Exception("my message"), then "my message" didn't appear in the console log, but only "Execution canceled".

After some debugging I found that normally the custom exception message will be printed in the console log as error, but not in the cancel-scenario, where the custom message will be silently ignored.

Maybe this should be mentioned in the developer's guide?

Best, Frank

4

Hi, looking into knime source org.knime.core.node.Node.java, lines 956-977 call NodeModel.execute():

try {
		// INVOKE MODEL'S EXECUTE
		// (warnings will now be processed "automatically" - we listen)
		rawOutData = invokeFullyNodeModelExecute(exec, exEnv, newInData);
} catch (Throwable th) {
...
		isCanceled = isCanceled || exec.isCanceled();
...
		if (isCanceled) {
				// clear the flag so that the ThreadPool does not kill the thread
				Thread.interrupted();

				reset();
				createWarningMessageAndNotify("Execution canceled");
				return false;
		} 
....

the method exec.isCanceled() is called, which by itself checks Thread.currentThread().isInterrupted; this causes isCanceled to be true and your exception to be ignored.