Compute overlapping area of different labels in labeling image

Dear all,

in scientific image processing workflow, a common task following image segmentation is to determine the amount of overlap between objects that were segmented from different sources, for example:

  • determining double-positive cells from segmentations of two markers, or
  • “sanitizing” ground-truth segmentations of time lapse images, such that the same objects have the same labels in different frames (whereas they might have a random label when the frames were segmented independently);
    (this second point classifies as a tracking problem, but is actually simpler than most tracking problems when the objects are large and the movement between frames is small)

In both cases, it’s not enough to simply determine if two labels overlap (a measurement provided by the Segment Features node and others), but we actually need to know by how much they do.

The Compare Segments node seems to be able to do this, but (as far as I understood) compares each target segment to all reference segments and so requires looping over frames e.g. when working with time lapse images.

I feel like I must be missing something.

Are there any example workflows available demonstrating the task of computing the pairwise overlap between two (or more) groups of label instances?

Hi @imagejan,

I have to admit that it took me some time to understand what you pointing out! :wink:

In order to understand what’s the matter, I have played around with the Labeling Arithmetic node that I thought would be able to solve this by merging the labels per pixel. But that only solves half of the problem, I reckon. In theory, converting the ImgLabeling with the merged labels to an ImgPlus and back (Labeling to Image and Image to Labeling) should leave you with one label per combination of original labels. You can measure those to compute the overlap but you are losing the information about the original labels…

In the end, this is not really helpful but it’s all the information that I could come up with for now. Out of curiosity: Is there functionality around this in in imglib2 / imglib2-roi? I have a hunch that it might be focussed on iteration a specific label instead of combinations of labels as well?

Best,
Stefan

Hi @stelfrich,

sorry for the delayed follow-up.

Sorry if I haven’t been very clear in my original post. Judging from your reply, you understood exactly what I was looking for :slight_smile:

In the end, this was exactly the missing piece for me, thanks for reminding me of that possibility to generate the “fragments” of overlapping labels.

At about the time of your post here, I also (re-)discovered the Compare Segments node, and it turns out that combining your approach with Compare Segments is the key to the problem here. So my strategy is as follows:

  • Generate the “fragment” label segments by doing the round-trip Labeling to Image and Image to Labeling, followed by Segment Features (also computing the size of each fragment),
  • Generate the original segments using the Segment Features node,
  • Use the Compare Segments node to determine all overlapping original labels for each fragment, and finally,
  • Use Ungroup, Joiner and GroupBy to generate the (symmetrical) confusion matrix of pixel overlaps for each pair of labels.

Here’s a screenshot:

I shared this as a component node on KNIME Hub (in case someone else might find this helpful):


No, there isn’t currently. imglib2-roi has seen a lot of development in recent years, and the possibility to combine and intersect ROIs from various sources easily is great, but this specific use case wasn’t covered yet. I submitted a pull request adding some utility methods to this end:


But I have to say, working with ImgLabelings directly in imglib2 is a bit cumbersome and so far required quite a lot of digging in the API etc., that was one reason for me in the past to prefer working with KNIME when I have to deal with image segmentations containing (potentially) overlapping objects :slight_smile:

2 Likes