Writing buffer: not enough disk space

KNIME Analytics Platform
1

Hi all,

I'm getting the following error when processing an image dataset of several 100GB in KNIME Analytics Platform:

  Execute failed: Error while writing to buffer, failed to write to file "knime_container_20160301_3895889446443796614.bin.gz": There is not enough space on the disk

I noticed that KNIME (on Windows) seems to write its buffer to the C: drive, which in my case is a limited-capacity SSD. I configured KNIME to use a workspace directory on another drive (D:), but how can I configure it to also save its buffer to another drive? Does it respect any environment variable that I could modify? I didn't find any documentation on that.

(Something similar was reported in this forum thread, but the solutions proposed to change the workflow rather than the configuration.)

I'd be thankful for any pointers.

 

Cheers,

Jan

 

2

Hi Jan,

you probably have to set another temp-directory location. This can be done via "File -> Preferences -> KNIME -> Directory for temporary files"

Hope that helps.

Best,

Martin

3

Also, I suggest to Loop over your data if possible (e.g. by using the Chunk Loop Start / End pattern or with Simple Streaming). Like this, only the last node is persisted and not all the intermediate results, which you potentially don't need anymore.

4

File -> Preferences -> KNIME -> Directory for temporary files

Thanks, I somehow missed that configuration option.

 

I suggest to Loop over your data if possible

Yep, I doing this now. Actually, it would be great to be able to inspect the output of a loop end node (i.e. the partially filled table) while it is running. Is something like this possible?

 

 

 

5

Yep, I doing this now. Actually, it would be great to be able to inspect the output of a loop end node (i.e. the partially filled table) while it is running. Is something like this possible?

As far as I know it's on the list. Don't know when it will be implemented, though. I will point the developers on this topic. Thanks as usual for the valuable feedback :-)

1 Like