Image Handling very slow

Hello again :slight_smile:

I have another problem using the Image Processing Extension:
I have a workflow with only one node: An Image Reader. Within this reader there are about 40.000 28 by 28 images loaded; JPEG. (MNIST Train Set).

As soon as I try to save the workflow the program will basically freeze with “Saving workflow… (cannot be cancelled) dialog”

The images itself are only about 22MB of data. There is absolutely no visual feedback while its saving. I cannot interact, I cannot cancel. It takes about 10 Minutes to save the workflow.

In addition, when trying to open the Node Configuration Dialog, the whole interface will freeze for 1 or 2 minutes; the Log Output will shout, that it noticed a potential DeadLock .

WARN KNIMEApplication$4 Potential deadlock in AWT Event Queue detected. Full thread dump will follow as debug output.
WARN KNIMEApplication$4 Potential deadlock in AWT Event Queue detected. Full thread dump will follow as debug output.

Is there some work done on these modules to increase handling of more than a handful files? If needed I could upload the files.

Hi @mereep,
There is an overhead associated with saving each image, this seems to greatly dominate the saving time in your case. You can try setting the maximum file size for data storage to a larger value (e.g 50mb):

.

best,
Gabriel

1 Like

hi thx for the reply.
I changed that setting, but it didnt influence the save-time considerably

You can try using the Don’t Save Start/End nodes and / or work with a smaller subset of the data until your model is ready to be tested, then run without saving the images.

best,
Gabriel

1 Like

Thx, thats what I am basically doing right now. But it feels a bit unsatisfying :wink:

1 Like