crash in Parallel Chunk End (multiple port)

KNIME Analytics Platform
1

java.lang.NullPointerException in workflow in screenshot below.
See trace back.

48
481914×1020 197 KB

2018-09-04 13:52:38,156 : DEBUG : KNIME-Worker-37 : LocalNodeExecutionJob : Parallel Chunk End (multi port) : 0:30 : Parallel Chunk End (multi port) 0:30 Start execute

2018-09-04 13:52:38,657 : DEBUG : KNIME-Worker-37 : Node : Parallel Chunk End (multi port) : 0:30 : reset

2018-09-04 13:52:38,657 : ERROR : KNIME-Worker-37 : Node : Parallel Chunk End (multi port) : 0:30 : Execute failed: java.lang.NullPointerException

2018-09-04 13:52:38,657 : DEBUG : KNIME-Worker-37 : Node : Parallel Chunk End (multi port) : 0:30 : Execute failed: java.lang.NullPointerException

java.util.concurrent.ExecutionException: java.lang.NullPointerException

at org.knime.core.util.ThreadPool.runInvisible(ThreadPool.java:617)

at org.knime.core.node.workflow.virtual.parallelchunkend.ParallelChunkEndMultiPortNodeModel.execute(ParallelChunkEndMultiPortNodeModel.java:161)

at org.knime.core.node.NodeModel.executeModel(NodeModel.java:567)

at org.knime.core.node.Node.invokeFullyNodeModelExecute(Node.java:1173)

at org.knime.core.node.Node.execute(Node.java:960)

at org.knime.core.node.workflow.NativeNodeContainer.performExecuteNode(NativeNodeContainer.java:561)

at org.knime.core.node.exec.LocalNodeExecutionJob.mainExecute(LocalNodeExecutionJob.java:95)

at org.knime.core.node.workflow.NodeExecutionJob.internalRun(NodeExecutionJob.java:179)

at org.knime.core.node.workflow.NodeExecutionJob.run(NodeExecutionJob.java:110)

at org.knime.core.util.ThreadUtils$RunnableWithContextImpl.runWithContext(ThreadUtils.java:328)

at org.knime.core.util.ThreadUtils$RunnableWithContext.run(ThreadUtils.java:204)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at org.knime.core.util.ThreadPool$MyFuture.run(ThreadPool.java:123)

at org.knime.core.util.ThreadPool$Worker.run(ThreadPool.java:246)

Caused by: java.lang.NullPointerException

at org.knime.core.node.workflow.virtual.parchunk.ParallelizedChunkContentMaster.nrExecutingChunks(ParallelizedChunkContentMaster.java:151)

at org.knime.core.node.workflow.virtual.parallelchunkend.ParallelChunkEndMultiPortNodeModel$1.call(ParallelChunkEndMultiPortNodeModel.java:184)

at org.knime.core.node.workflow.virtual.parallelchunkend.ParallelChunkEndMultiPortNodeModel$1.call(ParallelChunkEndMultiPortNodeModel.java:1)

at org.knime.core.util.ThreadPool.runInvisible(ThreadPool.java:615)

… 14 more

2018-09-04 13:52:38,657 : DEBUG : KNIME-Worker-37 : WorkflowManager : Parallel Chunk End (multi port) : 0:30 : Parallel Chunk End (multi port) 0:30 doBeforePostExecution

2018-09-04 13:52:38,657 : DEBUG : KNIME-Worker-37 : NodeContainer : Parallel Chunk End (multi port) : 0:30 : Parallel Chunk End (multi port) 0:30 has new state: POSTEXECUTE

2018-09-04 13:52:38,657 : DEBUG : KNIME-Worker-37 : WorkflowManager : Parallel Chunk End (multi port) : 0:30 : Parallel Chunk End (multi port) 0:30 doAfterExecute - failure

2018-09-04 13:52:38,657 : DEBUG : KNIME-Worker-37 : NodeContainer : Parallel Chunk End (multi port) : 0:30 : URI to Port 0:28 has new state: CONFIGURED

2018-09-04 13:52:38,657 : DEBUG : KNIME-Worker-37 : NodeContainer : Parallel Chunk End (multi port) : 0:30 : Output Folder 0:6 has new state: IDLE

2018-09-04 13:52:38,657 : DEBUG : KNIME-Worker-37 : NodeContainer : Parallel Chunk End (multi port) : 0:30 : URI to Port 0:31 has new state: CONFIGURED

2018-09-04 13:52:38,657 : DEBUG : KNIME-Worker-37 : NodeContainer : Parallel Chunk End (multi port) : 0:30 : Output Folder 0:32 has new state: IDLE

2018-09-04 13:52:38,657 : DEBUG : KNIME-Worker-37 : Buffer : Parallel Chunk End (multi port) : 0:30 : Using table format org.knime.core.data.container.DefaultTableStoreFormat

2018-09-04 13:52:38,657 : DEBUG : KNIME-Worker-37 : Node : Parallel Chunk End (multi port) : 0:30 : reset

2018-09-04 13:52:38,657 : DEBUG : KNIME-Worker-37 : Node : Parallel Chunk End (multi port) : 0:30 : clean output ports.

2018-09-04 13:52:38,657 : DEBUG : KNIME-Worker-37 : NodeContainer : Parallel Chunk End (multi port) : 0:30 : Parallel Chunk End (multi port) 0:30 has new state: IDLE

2018-09-04 13:52:38,658 : DEBUG : KNIME-Worker-37 : Node : Parallel Chunk End (multi port) : 0:30 : Configure succeeded. (Parallel Chunk End (multi port))

2018-09-04 13:52:38,658 : DEBUG : KNIME-Worker-37 : NodeContainer : Parallel Chunk End (multi port) : 0:30 : Parallel Chunk End (multi port) 0:30 has new state: CONFIGURED

2018-09-04 13:52:38,658 : DEBUG : KNIME-Worker-37 : Node : URI to Port : 0:28 : Configure succeeded. (URI to Port)

2018-09-04 13:52:38,658 : DEBUG : KNIME-Worker-37 : Node : URI to Port : 0:31 : Configure succeeded. (URI to Port)

2018-09-04 13:52:38,658 : DEBUG : KNIME-Worker-37 : NodeContainer : Parallel Chunk End (multi port) : 0:30 : PeptideIdentification_Parallelised_20180904 0 has new state: IDLE

2018-09-04 13:52:38,658 : DEBUG : KNIME-WFM-Parent-Notifier : NodeContainer : : : ROOT has new state: IDLE

2

Hi @lars20070,

Any chance you can provide a workflow + data to reproduce the problem? This would our live much easier finding (and fixing :crossed_fingers:) the problem.

Cheers,

Christian

3

@christian.dietz A minimal example does not crash.
https://www.dropbox.com/s/ozi3ew0546e0u6c/ParallelChunkNode_20180904.zip?dl=1

1 Like
4

@christian.dietz I constructed another example and will give it to you tomorrow on USB.
Good news: The crash is reproducible.
Bad news: The example is not very minimal.

5

@AlexanderFillbrunn @christian.dietz The null pointer exception is not specific to Parallel Chunk End (multiple port). The crash also occurs in standard Parallel Chunk End nodes.

03
033360×2100 1.1 MB

6

IMO, it would be best to parallelise ZipLoop.

1 Like
7

Hi Lars,

Thanks for the detailed feedback. We’ll take a look and follow up here!

Have a great weekend,

Christian