Convert Well-Known Text (WKT) To Labeling

Suppose I have a single channel image like this:

I already have a segmentation done for it. The segments are represented in well-known text (wkt) format like this:

Now I would like to use that as an input to create a “Labeling” that looks like this:

I think I would need the original image or at least the dimensions of the original image in order to map the polygons to a labeling of the correct size. But that should be doable, right? I also want the label ids in the “label_id” column match the pixel value the specific segment will carry.

Any suggestions? @imagejan , @stelfrich since I asekd the same question at the last NEUBIAS webinar, maybe you remember it and have a clue?

2 Likes

Hi @karbon_kreature,

This isn’t functionality that is supported out of the box and it will involve quite some work. Here are a couple of pointers from my initial investigation:

I assume that those results will not really be useful to you at the moment, but I will continue to look into this and take it step by step. The conversion from WKT to LabelingImg would also make for a great ImageJ2 plugin, IMHO.

Best,
Stefan

3 Likes

Hi @stelfrich,

thanks for your quick and informative response. I had a little look into it and I think I got what you are saying. Well, the next things we’ll do is specifically define what features we want from potential new nodes and then we can think about how this can be implemented…

If you say so, for sure.

Not for WKT => Labeling, but for the opposite direction I once implemented a little plugin:

  • Binary mask => polygon coordinates of the outline

For our use case, we had binary cell masks and required to get the coordinates of all the boundary pixels (in 2D).

The node is available via the FMI KNIME Plugins extension (which isn’t indexed on the KNIME Hub, hence the link to NodePit here):

The implementation – an ImageJ2 plugin – is here:

The output is not in ‘well-known text’ format, but two separate arrays for the X and Y coordinates, but converting this into WKT should be straight-forward.

The other way around can be achieved in a similar way, I guess.

2 Likes

Hi @imagejan,

that was indeed very helpful. I installed it and tried it out and also got the list of X and Y coordinates emitted by your ImageJ2 plugin. In general I also have a clear idea how to modify that plugin to do what I need. I’ll see if I can get it to work and will let you know. Thanks again for the quick response!

1 Like

Alright, I created my own Command Plugin named ContourPolygonBitmaskToWKT like this:

@Plugin(type = Command.class, headless = true, menuPath = "FMI>2D Binary Mask To Well-Known Text")
public class ContourPolygonBitmaskToWKT extends ContextCommand {

    @Parameter
    private OpService opService;

    @Parameter
    private Img<BitType> input;

    @Parameter(type = ItemIO.OUTPUT)
    private String output;

    @Override
    public void run() {
        Polygon2D polygon = (Polygon2D) opService.run(Ops.Geometric.Contour.class, input, true);
        int numVertices = polygon.numVertices();
        Coordinate[] coordinates = new Coordinate[numVertices + 1];
        for (int i = 0; i < numVertices + 1; i++) {
            Double x = polygon.vertex(i % numVertices).getDoublePosition(0);
            Double y = polygon.vertex(i % numVertices).getDoublePosition(1);
            coordinates[i] = new Coordinate(x, y);
        }
        GeometryFactory factory = new GeometryFactory();
        output = factory.createPolygon(coordinates).toText();
    }
}

I’m using the org.locationtech.jts:jts-core:1.18.1 library to create the polygon from a list of coordinates and then serializing the polygon using its build-in toText() method.

It creates exactly the output I wanted:

Now I need to go the opposite way. For that I have to find out how…

  • to instanciate a Polygon2D and
  • to get an Img<BitType> from that

Any hints would be appreciated. I suggest I submit a pull request once I’m done. Does that sound like a plan?

1 Like

GeomMasks has utility functions to create any kind of imglib2-roi ROIs.
You’ll then need Masks.toRealRandomAccessibleRealInterval and Views.raster to transform the ROI object into a Mask and then from there into an image (a RandomAccessibleInterval in ImgLib2 terms).

Here’s a Groovy script that illustrates this, runnable from Fiji’s script editor:

// Transform WKT into x and y double arrays, using JTS Geometry
@Grab('org.locationtech.jts:jts-core:1.18.0')

import org.locationtech.jts.io.WKTReader

polygonString = "POLYGON ((286 1, 287 1, 288 2, 289 2, 290 3, 291 4, 292 5, 286 1))"
geometry = new WKTReader().read(polygonString)
coords = geometry.getCoordinates()
x = new double[coords.length]
y = new double[coords.length]
for (i = 0; i < coords.length; i++) {
	x[i] = coords[i].getX()
	y[i] = coords[i].getY()
}

// Transform x and y into binary image, via ImgLib2-ROI
import net.imglib2.FinalInterval
import net.imglib2.roi.geom.GeomMasks
import net.imglib2.roi.Masks
import net.imglib2.view.Views

polygonRoi = GeomMasks.polygon2D(x,y)
mask = Masks.toRealRandomAccessibleRealInterval(polygonRoi)

result = Views.interval(Views.raster(mask), new FinalInterval([0, 0] as long[], [300, 300] as long[]))

Instead of the FinalInterval at the end, you can use your original Img (which is an Interval as well).

@imagejan: Super cool. Works almost the same in Java. I’ll wrap everything up and let you know. Nice, thanks a lot, that really helps a lot!

2 Likes

So, here is what I ended up doing:

@Plugin(type = Command.class, headless = true, menuPath = "FMI>Text To 2D Binary Mask")
public class ContourPolygonTextToBitmask extends ContextCommand {

    @Parameter
    private LogService logService;

    @Parameter(label = "X Dimension")
    private long xDim;

    @Parameter(label = "Y Dimension")
    private long yDim;

    @Parameter(label = "Well-Known Text Polygons")
    private String text;

    @Parameter(type = ItemIO.OUTPUT)
    private Img<BitType> output;

    @Override
    public void run() {
        GeometryFactory factory = new GeometryFactory();
        WKTReader reader = new WKTReader(factory);
        try {
            Polygon polygon = (Polygon) reader.read(text);
            List<RealLocalizable> points = Arrays.stream(polygon.getCoordinates())
                    .map(coordinate -> new RealPoint(coordinate.getX(), coordinate.getY()))
                    .collect(Collectors.toList());
            Polygon2D polygon2d = GeomMasks.polygon2D(points);
            RealRandomAccessibleRealInterval<BoolType> mask = Masks.toRealRandomAccessibleRealInterval(polygon2d);
            FinalInterval dims = new FinalInterval(new long[] {0, 0}, new long[] {xDim - 1, yDim - 1});
            IntervalView<BoolType> view = Views.interval(Views.raster(mask), dims);
            RandomAccessibleInterval<BitType> interval = Converters.convert(
                    (RandomAccessibleInterval<BoolType>) view, (in, out) -> out.set(in.get()), new BitType());
            output = new ImgView<>(interval, new ArrayImgFactory<>());
        } catch (ParseException e) {
            logService.warn(String.format("Could not create a polygon from input <{}>", text));
        } catch(Exception e) {
            logService.error(e);
        }
    }

}

Works pretty well and as expected. Only funny side effect is this:

As you can see, the original bitmasks on the left get translated to text just fine. But when I reconvert them to bitmasks again as you can see on the right, they are displayed a little differently. It doesn’t matter too much, because I still can use the GroupBy node to assemble an actual labeling like this:

segmentation

But it makes me wonder if it has some performance implications for larger images. It seemed as if the GroupBy node would execute faster using the original bitmasks compared to using the reconstructed “bigger” ones.

1 Like

Here is the corresponding branch on my fork on GitHub:

1 Like

Hi @karbon_kreature,

that’s awesome, thanks for sharing!

The bit masks generated by the Segment Features node (in the Bitmask column) are just as large as necessary to contain the bounding box of each segment. The positional information is saved in the image offset (that can be read/set by the metadata-handling nodes).

KNIME Image Processing (KNIP) uses the min value of an ImgLib2 Interval that can be at any offset.

Side note:
If I remember correctly, KNIP uses this in a slightly wrong way, since it allows a non-zero min for Img (instead of RandomAccessibleInterval) objects, despite the javadoc of Img explicitly stating:

An Img is a RandomAccessibleInterval that has its min at 0^n and its max positive.


You can probably do the same in your plugin by subtracting the minimum coordinates of each polygon from the interval bounds when creating the mask image, and then setting the min accordingly. I didn’t try it yet, though…


If you think it’s of general use and should be part of fmi-ij2-plugins, I’m happy to accept pull requests!

Definitely have done this from within a Java Snippet, but I also don’t see why it shouldn’t work in a Command.

Interesting observation, @imagejan. I guess it boils down to KNIP being very “relaxed” (not necessarily in a good way) about RAI vs Img, in general. Does native KNIP ever create Imgs with non-0 min or is this only an issue with the ImageJ2 integration?

I would argue that it is of general interest. Plus, it would really be a pity if this got lost on a branch…

1 Like

As far as I can tell, this is pretty omnipresent in the offset-handling of KNIP. For example, this method wouldn’t make any sense at all if Img objects would be treated to have always zero-min (as defined in the Img javadoc):

Here we go, my pull request! Please review and let me know if you want me to change anything. I wasn’t that sure about what to put in the initial javadoc at the beginning of the class files, but I hope what I did is fine. Thanks for encouraging me and thanks for all the help!
Create plug-ins to convert well-known text to bitmasks and vice versa #14

1 Like