We have been using KNIME to measure fiber orientation and variability. We have found the table view with embedded images to be very helpful, as we can sort the table by various metrics, and verify that the metrics agree with what we see in the images. And of course we can quickly see if a segmentation went wrong.
The only issue we’ve found is that sorting the table seems to take 2-4 minutes and it seems stuck at ‘50% Merging temporary tables’ for most of that time. The table has 200 rows. Each row has 3 images (original, thresholded and segmented), and ~10 numerical values. Intuitively I would of thought sorting this relatively small table would be faster. Are there any known issues with sorting tables?? Are there any tricks to make it faster??
Hi @bnorthan,
I have trouble reproducing this slowness but for me the Sorter node is quite fast (max 2 seconds) even if I have thousands of large-ish images. So I need a bit more information on your data / workflow:
How large are the images ? (It should not really make a differnce but it might)
Are you sorting by many attributes or by just one?
Are you able to share the workflow with me? If so either post it here or send it to me by email at gabriel.einsdorf@knime.com
The sorter node itself is also very fast for me. The issue occurs when I open a table and then click on the column header, then click ‘sort ascending’ (or descending).
Do you see the problem if you try sorting by column header in this way?? Or is it still fast for you?
I played around a bit with generated images and the sorting in the view does become slower when I put larger images in, but nothing close to your numbers, how big are your images?
My images are 1024 by 1024. The table has two hundred rows, and it also contains a thresholded images and a segmented image. So in the final table there are 3 1024 by 1024 images in each row.
All the Image Processing settings are still at default values (see screen shot). Should I change some of these to optimize table sorting performance?? The dialog says it’s recommended to set the file size to 0.
@christian.dietz Thanks for your quick reply. I’m travelling all this week but I’ll try and get you guys an example sometime early this week. Everything in this project can be open so eventually we will share all the workflows and even the entire VM we are using.
Sorry for the delayed reply. I was waiting to get a copy of the AWS VM that this project is on. I now have that and can share things with you guys. The images and workflows are both on the same VM. I can share the VM with you. Or I could start the VM on my AWS account and give you the password. Or alternatively I can put the images on drop box, and then share the exported workflow, in this case you’d to run the processing again, which would take a couple of hours.
Do you have the workflow in an executed state on the VM? If so, could you spin it up and use a Table Writer to write a copy of that table and upload that to e.g. Dropbox?
