Temp Files take enormous amount of space

Regarding the C-files: There is not much we actually can do about this, as Theano is creating these files. Maybe you can uninstall theano?

Regarding the other temporary files: In the current implementation of KNIME Image Processing, if you read or process an image, the data is duplicated for each image. The duplicated images are only written to disk in the temp folder if the used main memory exceeds a certain percentage of the heap space available to the JVM. That’s why you experience the temporary files only after a certain amount of images have been read. However, 200mb of data (even uncompressed) shouldn’t result in > 10GB of temporary data. This is something we will investigate.

Having all of that said, we’re working on a new version of KNIME Image Processing which actually doesn’t duplicate all the data all the time. Especially for very large images this is a problem which we will address. I can’t give you an estimate when it will happen, though.

1 Like

Sounds good. I cannot explain that also. As a matter of fact even very small images seem to be blown up by a big amount of data. An image of some KB becomes multiple MB of disk space.
I wanted to also look a bit into that, but I cannot get knip to work within my Eclipse for some reason :frowning:

If I can help in some way, let me know!

Thanks! Confirmed close Knime, ensure it is not running. And can move to a different disk space, or delete. Using Optane Drive and had accrued 95GB of temp directory files!

A post was split to a new topic: Temp files slowing KNIME