Parallel Chunk: Parallel chunks are not present anymore

Feedback & Ideas
1

Hi,

since recently the Parallel Chunk End node occasionally throws this error: Parallel chunks are not present anymore

But I cannot find or deduce any possible cause. Here is the log:

2025-02-24 10:09:05,258 : ERROR : KNIME-Worker-204-Parallel Chunk End 3:1946:0:1944:0:1939 :  : Node : Parallel Chunk End : 3:1946:0:1944:0:1939 : Parallel chunks are not present anymore.
org.knime.core.node.KNIMEException: Parallel chunks are not present anymore.
	at org.knime.core.node.KNIMEException.of(KNIMEException.java:124)
	at org.knime.core.node.message.Message.toKNIMEException(Message.java:221)
	at org.knime.core.node.workflow.virtual.parallelchunkend.ParallelChunkEndMultiPortNodeModel.execute(ParallelChunkEndMultiPortNodeModel.java:216)
	at org.knime.core.node.NodeModel.executeModel(NodeModel.java:596)
	at org.knime.core.node.Node.invokeFullyNodeModelExecute(Node.java:1284)
	at org.knime.core.node.Node.execute(Node.java:1049)
	at org.knime.core.node.workflow.NativeNodeContainer.performExecuteNode(NativeNodeContainer.java:603)
	at org.knime.core.node.exec.LocalNodeExecutionJob.mainExecute(LocalNodeExecutionJob.java:98)
	at org.knime.core.node.workflow.NodeExecutionJob.internalRun(NodeExecutionJob.java:198)
	at org.knime.core.node.workflow.NodeExecutionJob.run(NodeExecutionJob.java:117)
	at org.knime.core.util.ThreadUtils$RunnableWithContextImpl.runWithContext(ThreadUtils.java:369)
	at org.knime.core.util.ThreadUtils$RunnableWithContext.run(ThreadUtils.java:223)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
	at org.knime.core.util.ThreadPool$MyFuture.run(ThreadPool.java:123)
	at org.knime.core.util.ThreadPool$Worker.run(ThreadPool.java:246)

image
image1383×225 12.8 KB

Resetting and executing again seems to mitigate the issue but it’s quite annoying to manually intervene each time.

Best
Mike

2

Hey @mwiegand,

When does this occur? do you typically see it after running a workflow for a while? or is it sparse?

TL

3

Hey @thor_landstrom,

pretty much at first run after starting Knime. Once reset and retried, it works.

Cheers
Mike

1 Like
4

Hey @mwiegand,

Since you mention it only happens on startup, I think its probably due to the buffer being cleared and the chunk not being in there anymore. I made a ticket for it AP-24058.

I will let you know if I see any updates on it.

TL

3 Likes
5

Thanks Thor, let me know if I assist in additional debugging.

1 Like
6

Hi @mwiegand,

I looked into AP-24058 that @thor_landstrom opened. It is true that since recently (specifically 5.4.0) this “new” message appears when the ephemerally-created, parallel chunks are cleared.
image
However, that error shows up prior to 5.4.0 as well, but with a nasty, Java-specific error message (internal ticket AP-22860).

The chunks will not be present anymore if the Parallel Chunk Start node is done executing and is not reset. To (re-)use parallel chunks, you always have to execute the Parallel Chunk Start node and Parallel Chunk End nodes together. Did you do that after the starting the AP? If yes, and the error still appears, this is unexpected and a bug. However, I could not reproduce this scenario yet.

Best, Leon

1 Like
7

Replication is also difficult and I could not deduce a possible trigger so far. If I get an idea, I will update this topic.

2 Likes