Image Reader: tif from different folders

Hi,

I wanted to load tif images (16-bit, BD pathway, one images per file) from different folders. If I select only the two files from a folder it works nicely, but using the advanced option with file name column, then throws errors (see below) or has this strange behavior:

Input is "Location" from List Files node and only one file (/Volumes/ProMebs/121011_021PR_SVECclones_tubes/images/021PR121011A-SCVECclonesTubes-04h/2012-10-11_001/Well C003/GFP_lampB - n000000.tif). Then it reads all 48 images distributed in 24 folders (on level "Well...") and names the RowID with "Experiment.exp_0", "..._1" and so on, which is a text file in the folder above (2012-10-11_001). I have set to take the complete path & file name as row key. 

Why is the Reader reading 48 files instead of only one? Why this strange row key? How to know which images is which? By the way the tab Options and selected files is empty as is should.

At the end I wanted to use it with flow varible and open image after image. I couldnt get it running to read somple .tif from different directories and I dont like the idea to rename files and put them all in one folder.

Cheers,

Martin

ps: here are some errors that occured while I tried to set it up.

ERROR     ReadFileImgTable     An IO problem occured while opening the file /Volumes/ProMebs/121011_021PR_SVECclones_tubes/images/021PR121011A-SCVECclonesTubes-04h/2012-10-11_001/Well N003/GFP_lampB - n000000.tif (No Plate File)

ERROR     Image Reader     Execute failed: Encountered duplicate row ID  "GFP_lampB - n000000.tif" at row number 2

ERROR     NodeContainerEditPart     The dialog pane for node 'Image Reader 2:5:20' has thrown a 'RuntimeException'. That is most likely an implementation error.

ERROR     ReadFileImgTable     An IO problem occured while opening the file /Volumes/ProMebs/121011_021PR_SVECclones_tubes/images/021PR121011A-SCVECclonesTubes-03h/2012-10-11_000/Well K003/GFP_lampB - n000000.tif (Bad file descriptor)

Hi Martin,

we are currently a little bit confused... Of course we see the problem ;), but we cannot directly offer a solution. If it would be possible, could you provide us the complete "Well ..." folder? As far as I understand this is somehow the root folder for all your subfolders containing the images and these .txt files?! We could then redesign your workflow and hopefully figure out what's wrong.

Maybe BioFormats (which we use internally) makes use of these .txt files and reads in all files listed in this .txt file. You could filter all files which are not .tif in your list files node?

Anyway, it would be great if you could provide us the dataset!!

Cheers!

Christian

 

 

 

Hi Christian,

thanks a lot for the quick reply. Thanks awesome!

Well I dont think it is a bioformat thing, and actually these .txt files should be just ignored. Anyway, I cut down the number of images and zipped it. The folder struction is like this:

/experiment/barcode/acquisition/well/image1channel1.tif ...image1channel2.tif ...image2channel1.tif and so on...

I left everything as it comes from the microscope, but reduced the number of images. There are 2 barcodes, one has 2 acquisitions, with 2 wells; the other has 1 acquisitions, with 3 wells. There are always 1 image with 2 channels. Since the upload here is limeted to 8MB I put it for download here:

http://dl.dropbox.com/u/28983746/BD%20Pathway%20example.zip

...but will delete it there in two days.

As I said, I used the List Files to specify the images to be read...

Cheers,

Martin

 

Hi Martin,

thanks for the data, this helped a lot! I guess I figured out what's wrong. Two issues:

a) We had a bug: "Use complete path as rowkey" didn't work. Thank you for pointing us on that. I just fixed it. The changes will be available in about 30minutes on our nightly build and (hopefully) next week in the stable version with our release of 1.0.3!

b) I'm not an expert about data formats, but your tiff file somehow has some header information, such that BioFormats really wants to open all related files in the experiment. As both files exist in the table of ListFiles (e.g. GFP_lampB - n000000.tif from Well C003 and GFP_lampB - n000000.tif from C004) they both want to open their linked file and therefore a file will be loaded twice. Additionally there is an "experiment.exp" file in each base folder (e.g. 2012-10-11_001). This file also tries to load the tiffs. So actually they are loaded three times ;)

Solution for now:

You could simply restrict your List Files nodes to the .exp (Just set: Expression = "Exp" and check "File Extension" in the configure of List Files). But: This will only work if any of your experiment folders contains all required files, especially .plt files + .dye files (e.g. in 2012-10-11_001 these files are available, in _000 they aren't -> Error: No Plate File).

Additionally I will ask Melissa from BioFormats how we could manage it that only the tiffs are read in and not there "partner files" or "linked files". Maybe we can add a checkbox or use the "load group files" field in the advanced settings of the image reader (which actually exactly does what we need, but somehow not with this format).

I will keep you updated!

Cheers,

Christian