Image Handling very slow

KNIME Extensions
1

Hello again :slight_smile:

I have another problem using the Image Processing Extension:
I have a workflow with only one node: An Image Reader. Within this reader there are about 40.000 28 by 28 images loaded; JPEG. (MNIST Train Set).

As soon as I try to save the workflow the program will basically freeze with “Saving workflow… (cannot be cancelled) dialog”

The images itself are only about 22MB of data. There is absolutely no visual feedback while its saving. I cannot interact, I cannot cancel. It takes about 10 Minutes to save the workflow.

In addition, when trying to open the Node Configuration Dialog, the whole interface will freeze for 1 or 2 minutes; the Log Output will shout, that it noticed a potential DeadLock .

WARN KNIMEApplication$4 Potential deadlock in AWT Event Queue detected. Full thread dump will follow as debug output.
WARN KNIMEApplication$4 Potential deadlock in AWT Event Queue detected. Full thread dump will follow as debug output.

Is there some work done on these modules to increase handling of more than a handful files? If needed I could upload the files.

saving_knime
saving_knime1277Ă—679 26.3 KB

2

Hi @mereep,
There is an overhead associated with saving each image, this seems to greatly dominate the saving time in your case. You can try setting the maximum file size for data storage to a larger value (e.g 50mb):

Selection_031
Selection_031926Ă—836 102 KB
.

best,
Gabriel

1 Like
3

hi thx for the reply.
I changed that setting, but it didnt influence the save-time considerably

4

You can try using the Don’t Save Start/End nodes and / or work with a smaller subset of the data until your model is ready to be tested, then run without saving the images.

best,
Gabriel

1 Like
5

Thx, thats what I am basically doing right now. But it feels a bit unsatisfying :wink:

1 Like
6

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.