blob and buffer errors

I am using the latest KNIME version on the Mac (latest OSX) and keep getting errors on nodes such as these below which stops the node from completing. Any ideas what the problem is;
ERROR X-Aggregator 0:204:1 Execute failed: Error while writing to buffer, failed to write to file “knime_container_20200810_8568262710548603014.bin.snappy”

and…

ERROR X-Aggregator 0:193:1 Execute failed: Error while writing to buffer: Blob file location “/Applications/KNIME 4.2.0.app/Contents/MacOS/col_1/000/000” does not exist

and…

Execute failed: Error while writing to buffer, failed to write to file “knime_container_20200810_7894528850755034336.bin.snappy”: /private/var/folders/dx/lggccpg…

Any ideas. I have it happen on other nodes too.

Simon.

Not sure whether this is actually related, but I had many, many problems with these “snappy” files as well. Suggest to check here if it might be related:

At the end, changing the storage format to GZIP fixed the issue (details very last post in the thread):

3 Likes

Thanks for this I will try this out and let you know if the problem goes away.

Simon.

1 Like

Fingers crossed, no issues yet since making the compression setting in the ini file.

Thanks,

Simon.

2 Likes

Sadly, the issue is still present. I am not sure what is causing it.
I usually have to keep resetting the workflow right back to the beginning and starting it again.
A real pain!!

Simon.

Hi @richards99 -

Sorry for the trouble. Do you have a workflow that will reliably reproduce this issue - or is it something that only happens intermittently, which is then fixed by a workflow reset?

Unfortunately it happens in any workflow and on any node.
Some days it will be fine and then other days the issue returns.
A node won’t run complaining about blobs, and I can keep resetting earlier nodes and they then won’t run either despite having run earlier. In the end I have to reset the entire workflow.
It could be coincidence but the error usually starts in a meta node or on a loop end.
I’ve never encountered this before, as I’ve always used it on a PC before. It’s only now I’m using knime on a Mac and encountering these issues.
I’m using the default workspace Knime suggests on install, and the only modification I’ve done is to increase memory to 10Gb, the Mac has 16Gb altogether and it is a 2019 MacBook Pro.

Simon.

Thanks for the additional info. I’ll see if I can get some developer input here.

1 Like

Hi @richards99,

Thanks for reaching out and bringing this issue to our attention. Since you are pointing out that once the issue occurs in some node, any other nodes in the workflow cannot be successfully executed any more either, I think what is happening is that the temporary workflow directory associated with the workflow is deleted while the workflow is still running. We have only seen this issue on Mac and I wonder if it is related to how Macs manage temporary files.

Anyways, to try and diagnose the problem, I have a few questions for you:

(1) In your KNIME log (or, the next time the error occurs, in your AP GUI) do you find (and can you share) the full stack trace of the errors you are experiencing? The stack trace is the part after the ERROR that looks like this

java.lang.NullPointerException
	at java.util.ArrayDeque.addLast(ArrayDeque.java:249)
	at org.xerial.snappy.buffer.CachedBufferAllocator.release(CachedBufferAllocator.java:77)
    ...

or this

java.lang.RuntimeException: Cannot read file "knime_container_20200301_1905559478659870280.bin.snappy"
	at org.knime.core.data.container.DefaultTableStoreReader.iterator(DefaultTableStoreReader.java:126)
    ...

(2) Did switching the compression format do anything? Did the errors change? Were fewer errors logged?

(3) When running into the error, can you check whether the temporary workflow directory, to which tables or blobs were written, still exists? In the case of the second error you reported, that would have been /private/var/folders/dx/lggccpg…

(4) For about how long were the Analytics Platform and the workflow open before the error occurred (i.e., was it openened recently or had the AP been running and the workflow been open for a few hours, days, …)?

Thanks,

Marc

2 Likes

Hi marc-bux,
Sadly, the compression change did not make any difference to the frequency of the errors or the type of errors.
I have just had an error today, in which the workflow had only been loaded for 1hr, most of it ran fine, and then towards the end I had the same issues with this error:
ERROR X-Aggregator 4:187:1 Execute failed: Error while writing to buffer: Blob file location “/Applications/KNIME 4.2.0.app/Contents/MacOS/col_1/000/000” does not exist
When I tried to go to this directory, it did not exist, only this directory existed:
/Applications/KNIME 4.2.0.app/Contents/MacOS/
And it contains a small 47Kb file called KNIME which it says is Unix Executable.
If I close the workflow and then reload it, and then try and run the workflow, it still fails complaining about:
Execute failed: Error while writing to buffer: Blob file location “/Applications/KNIME 4.2.0.app/Contents/MacOS/col_1/000/000” does not exist
In this instance I tried to reset the workflow a few nodes before the fault, and then rerun it, but it still fails at the same position. I then decided to reset the workflow about half way (so quite a lot of nodes earlier) and then reran it, and the workflow was then successful.

Simon.

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.