loop workflow for loading Images

KNIME Extensions Image Processing
#1

Hi it’s me again :)

 

I’m working right now on a loop workflow for reading in Images (e.g. tif, czi). Right now I am stuck and I wanted to ask if it’s at least theoretical possible what I am trying to achieve.

My intension is that when you perform very long term experiments, like 3+ days you’re getting a lot of data from your Microscope. So after waiting 4 days for the experiment to be finished you can start your KNIME workflow and analyze the images. That will cost again maybe many hours or even a few days. Now I’m trying to shorten that.

Let’s say you have a folder which is continuously filled with the Images from the Microscope software (e.g. 50 Images per minute).

Is it possible to create a KNIME workflow which is continuously checking that folder for new Images (e.g. every minute or so), loads the new ones in, performs some Image analysis (e.g. Segmentation, Watershed,...), and collects the results (adding the new columns to a table). And maybe even seeing the results live while the experiment/workflow is running  ???

Any help would be great.

 

Thanks a lot.

Flo

#2

Hi Flo,

Martin and me were also curios if this is doable ;-) So we decided to give it a try and compose such a workflow: The good new is: Yes, it works. The more or less bad news is: It involves a little trick.

Attached you find the workflow. Just put whatever you want after the image reader and connect it to the in-ports to which the image reader is currently connected. If you run the workflow it will run forever (to be precise it runs min-number iterations, see configuration of loop end). Now the tricky part: If you want to stop the workflow you have to delete the file "DELETE_ME" in the directoy you specified (see annotations for details). We are pretty sure there will be more elegant ways to do all this stuff in the future, but hey... it works and does hopefully exactly what you want.

If you face any problems, let us know. This was really fun creating the workflow and we are happy to improve it :-)

Greetings,


Martin and Christian

 

#3

Now that’s a workflow :D

I’ll need some time to work this through but thanks for the enormous effort.

 

Have a nice day.

Flo

#4

Hi Flo,

does the workflow do the job? You can test it as follow:

  • Create an empty directory.  Configure the first meta-node such that it uses this directory.
  • Start the workflow (DELETE_ME file should appear in the directory)
  • Put any images in the loop (right-click on loop end).
  • Pause the workflow and have a look at the table-cell-viewer in the loop
  • All images should be read in.
  • Restart the workflow
  • delete "DELETE_ME" file in the folder.
  • The workflow should stop and all the images should appear at the loop end.

Hope this works ;-)

Christian

Btw: you need the update your KNIME with the Quickforms (using KNIME Update-Site)

#5

Hi  Flo and Cristian;

If you are using a Mac with OSX or any UNIX Operating System there is a way to make an script that fills and clean up a directory.

I made this short research for this matter, I have not try any of the next scripts yet, but I made somo of that years ago, somo of them must work.

I hope this info could be usefull, just let us know if any othis works...

1. Dynamic Folder Workflows

1. How to set automator (mac) to create folders and fill them with x number of files?

2. Apple Support Communities

3. Example Workflow 3: Automate Podcast Preparation

4. How to Schedule Automator Tasks in Mac OS X

5. 10 Awesome Uses for Automator Explained

#6

Thanks karelman I have also a Mac so I will look into that too.

 

Martin and Christian,

it seems to work. All Images I am putting into the folder are loaded and analyzed. But I can’t spot the delte_me file. It should be in the same folder I am using for the Images right ? It does not appear :(

#7

Hi Karelman,

I will take a look! Thanks!

Christian

#8

Hi Flo,

do some of the nodes in the workflow complain about something? can you check node CREATE MARKER FILE and maybe replace "directory" with manual path and see if a file was created?

Seems to be FileSystem related. I will try it on a MAC as soon as possible!


Christian

 

#9

It is working now and it was my own dumb fault :-D

As String Value I typed in D:\New folder which was wrong I had to type in D:\New folder\ with the \ 

 

Thanks a lot for the great workflow. I would have never been able to get it done. You guys are awesome :-)

 

Greetings,

Flo