Creating Workflow to assemble XYZ Stacks

Hi,

I have been trying to assemble a coupe of thousand images by ImageJ Macro and it's more than bulky and difficult to debug, therefore I want to try doing it with knime.

I have the following starting point: the Microscope creates a folder full of 3-channel colour images. Only one channel is used per image (so red/blue/green aquisition generates 3 images with 3 channels each, two of which are empty).

X, Y and Z position, as well as the contained channel are encoded in the file name. Example:

20151110_SampleA_R00C00_CH1_Z00000.tif (top/left image, channel 1, first image in Z-Stack)

I can load the images with ImageReader Node. Nex step (before calculating a full focus of each position) would be to put each z-Stack in a row for itself with "Colum to Grid" (?) and then use the "stack Focus" macro from ImageJ on each Row, I'd suggest. This is the first point where I am lost (my KNIME training was too long ago), how do I make it recognize the number of Z-positions and convert the table? Rename the Images? Only read one Stack (how?) and loop until I got all images?

 

 

Hi Martin,

can you upload a small example data-set? Then I'm sure we can send you an example workflow which helps you out.

 

Cheers,

 

Christian

Thank you!

2x2x2 Stack with 3 channels (200MB zip)

https://bwsyncandshare.kit.edu/dl/fi6JWC3J6o4jULEHbwYbqn4s/.zip

Hi Martin,

https://depot.uni-konstanz.de/cgi-bin/exchange.pl?g=m74lxrwtz5

This workflow creates for each Row-Column-Position a 3 channel and 2 z-slices stack.

It also filters all images which have a missing channel. In this data set 20150522_A__R00C01_CH2_Z00000.tif is missing.

 

Cheers,

Tim-Oliver

Hi Tim,

thank you a lot for the workflow. Upon loading it I get an "Missing Image Reader" alert and a placeholder. The providing Plug-in (org.knime.knip.io) is installed and up to date (1.2.1.201409131850 is the latest the updater gives me from my sources). Do you use a newer plug-in?

The missing file must have been an uploading glitch, sorry for that.

Bests

Martin

Do you use KNIME 3.0 already? I think Tim created the workflow with KNIME 3.0 and KNIP 1.4.1. You can download KNIME 3.0 from knime.org and install the Image Processing Extensions via File -> Install Extensions -> Trusted Community Contributions.


Does this work?

Hi Martin,

KNIME 3.0 was recently released as well as a new version of KNIME Image Processing (1.4.x) with many new features.

Head over to the download page  and get it while it is stil hot.

Greetings,

Gabriel

Downloaded and it found the right Node. Was too shy to install it because the description said: not yet all nodes available for Knime 3 and "early access". Will let you know how it works out and upload the workflow if you like.

@Tim-Oliver

What is supposed to be in Name_Arr[1..4]? It seems something is not working with a full set of images (maybe because the filename is longer and has other things delimited by "_" in it?). First Warning sign ist with Column Rename inside name processing: The following columns are configured but no longer exist: "Name_Arr[0]_Arr[1]". Then group by fails and...

I see that it does generate substrings from the name, depending on the position in the filename. Can't use it with fixed substring positions (the _R00C00_Z00000 part is fixed, but users may rename everything before that). Does something like "indexOf($name,"R00C00")" work in Knime to detect the position of the row / column / z-Stack position?

EDIT: Found it. What did you intend to cut into the string? I managed to cut the Name to "R00C00_CH1_Z00000.tif". Somehow the Split in Row and Colum is not working, output is "0" and "0" for the first stack of images (as intended?) and the second stack has "201522" and "?" in the table...

Yes this stack reconstruction is based on a naming pattern. If the naming pattern changes the "name processing" should be adapted.

What the workflow does:

Input to MetaNode "name processing": R00C00_CH1_Z00000.tif

String Manipulation: R00C00_CH1_Z00000 (remove ".tif")

Cell Splitter: Split at "_" returns 3 columns with values: "R00C00", "CH1", "Z00000"

Cell splitter: Split "R00C00" at "C" returns 2 columns with values: "R00" and "0" (<- 0 is typed on int)

Apply some filter and rename: Result columns: "Channel", "Z", "Row", "Col"

With the 3 String Manipulation nodes I remove the letters from the values in the columns "Channel", "Z" and "Row". Then I convert the String into Numbers.

If your filenames change in length or pattern than you have to change the settings of most likely every node in this meta node.

This reconstruction only works if all images are named by the same pattern.

I managed to cut "R00C00_CH1_Z000000" by substr($name$, indexOf($name,"R00C00"), 16. This cuts the substring from wherever it starts in the file name :) Fortunately the R...C...CH... is appended by the microscope and beyond user control :) all images are named the same user defined string plus appended position, channel, z-stack information.

Result of 2nd cell Splitter is Name (as I cut from string), 00C00 / CH1 / Z00000 / 0 / 0.

After 2nd String Manipulation Channel, Z Row Col are the names of the Cloumns and they are containing only numbers.

However: This only works for the first 6 images, then it goes haywire and puts other parts of the file name into the columns... and I got no idea why. This goes wrong from the first string manipulation. The Image Properties (appending the filename to each row) looks homogenious in output and ok.

Can you send me your current workflow?

Than I take a quick look at it.

https://bwsyncandshare.kit.edu/dl/fiHbACyeVKpr17cmNkPzXc4V/ImportImgs_Universal.zip

What I don't get is why it works correctly for the first 5-6 images and then fails. Did I miss some configuration of the first Node inside "name processing"?

Can you send me the images? Or included data during export :) There is a checkbox (exclude data) which is selected by default in the export dialog.

The names of the images are always something like this: "custom-string_R01C00_CH2_Z00001.TIF"

To extract just the "R01C00_CH2_Z00001" part of this name we can use this

reverse(substr(reverse($Name$), 4, 17))

in the first String Manipulator Node.

I think I know what went wrong: I am looking for "R00" with the indexOf function to look where the constant, position defining string starts. Of course that can only work on images containing R00. As soon as R01 starts it all goes to hell. (It was working fine in my ImageJ Macro, because I could look for this position once, then apply the same borders to all images). Is there a way to look for the position of R00 in the first image (or better even: in the first image actually containing it) and apply this number to all substring operations on the table?

Would it help if I uploaded the .ijm file that is working?

Here is a test-set of images with workflow anyway. Thanks for the great support!

https://bwsyncandshare.kit.edu/dl/fiHbACyeVKpr17cmNkPzXc4V/ImportImgs_Universal.zip

EDIT: with your suggestion to crop it from the other side (reverse) it's running right now (I can see the loop running).

I am happy to hear that it is running now :)

 

I don't understand what you mean with "apply the same borders to all images". This workflow only reconstructs the image stacks.
 

I was talking about the limits of which part of the name string to crop. My approach was to interactively look for the substring "R00" because it marks the beginning of the part of the file name containing the relevant information, no matter what the user does to the images (name them .tif or .tiff or whatever string in the beginning he/she chooses). Of course that can only work for images that contain R00. Instead of some kind of error / warning all the images containing R01 and R02 for the following rows got cropped from position 0 (very beginning of the name string) and I was confused why it worked for some but not all images...

It's a little less flexible than my approach but working fine. Will have to figure out how to re-wire it for assembly only 2 of 3 channels and from what I understood your workflow just merges the Z-Stack? I will use the full-focus macro from ImageJ, it takes the "most in focus" parts of each image in the stack and combines it into one image. Thank you a lot. Will upload the full workflow some time next week.

Have a nice weekend