I think I would need the original image or at least the dimensions of the original image in order to map the polygons to a labeling of the correct size. But that should be doable, right? I also want the label ids in the “label_id” column match the pixel value the specific segment will carry.
Any suggestions? @imagejan , @stelfrich since I asekd the same question at the last NEUBIAS webinar, maybe you remember it and have a clue?
This isn’t functionality that is supported out of the box and it will involve quite some work. Here are a couple of pointers from my initial investigation:
In order to create the path we’ll either need to get individual coordinates from the WKT representations of polygons and create it ourselves or use a library, e.g. Geometry — GeoTools 30-SNAPSHOT User Guide
I thought we could easily use this library then after installing the extensions, but I couldn’t figure it out yet
Once we have a GeneralPath, we can create a ROI from it and should be able to convert this into a LabelingImg for use in KNIME
Please note that the labeling very likely will be just the boundary and not all the pixels inside of it (it I am not mistaken)
I assume that those results will not really be useful to you at the moment, but I will continue to look into this and take it step by step. The conversion from WKT to LabelingImg would also make for a great ImageJ2 plugin, IMHO.
thanks for your quick and informative response. I had a little look into it and I think I got what you are saying. Well, the next things we’ll do is specifically define what features we want from potential new nodes and then we can think about how this can be implemented…
Not for WKT => Labeling, but for the opposite direction I once implemented a little plugin:
Binary mask => polygon coordinates of the outline
For our use case, we had binary cell masks and required to get the coordinates of all the boundary pixels (in 2D).
The node is available via the FMI KNIME Plugins extension (which isn’t indexed on the KNIME Hub, hence the link to NodePit here):
The implementation – an ImageJ2 plugin – is here:
The output is not in ‘well-known text’ format, but two separate arrays for the X and Y coordinates, but converting this into WKT should be straight-forward.
The other way around can be achieved in a similar way, I guess.
that was indeed very helpful. I installed it and tried it out and also got the list of X and Y coordinates emitted by your ImageJ2 plugin. In general I also have a clear idea how to modify that plugin to do what I need. I’ll see if I can get it to work and will let you know. Thanks again for the quick response!
Alright, I created my own Command Plugin named ContourPolygonBitmaskToWKT like this:
@Plugin(type = Command.class, headless = true, menuPath = "FMI>2D Binary Mask To Well-Known Text")
public class ContourPolygonBitmaskToWKT extends ContextCommand {
@Parameter
private OpService opService;
@Parameter
private Img<BitType> input;
@Parameter(type = ItemIO.OUTPUT)
private String output;
@Override
public void run() {
Polygon2D polygon = (Polygon2D) opService.run(Ops.Geometric.Contour.class, input, true);
int numVertices = polygon.numVertices();
Coordinate[] coordinates = new Coordinate[numVertices + 1];
for (int i = 0; i < numVertices + 1; i++) {
Double x = polygon.vertex(i % numVertices).getDoublePosition(0);
Double y = polygon.vertex(i % numVertices).getDoublePosition(1);
coordinates[i] = new Coordinate(x, y);
}
GeometryFactory factory = new GeometryFactory();
output = factory.createPolygon(coordinates).toText();
}
}
I’m using the org.locationtech.jts:jts-core:1.18.1 library to create the polygon from a list of coordinates and then serializing the polygon using its build-in toText() method.
GeomMasks has utility functions to create any kind of imglib2-roi ROIs.
You’ll then need Masks.toRealRandomAccessibleRealInterval and Views.raster to transform the ROI object into a Mask and then from there into an image (a RandomAccessibleInterval in ImgLib2 terms).
Here’s a Groovy script that illustrates this, runnable from Fiji’s script editor:
// Transform WKT into x and y double arrays, using JTS Geometry
@Grab('org.locationtech.jts:jts-core:1.18.0')
import org.locationtech.jts.io.WKTReader
polygonString = "POLYGON ((286 1, 287 1, 288 2, 289 2, 290 3, 291 4, 292 5, 286 1))"
geometry = new WKTReader().read(polygonString)
coords = geometry.getCoordinates()
x = new double[coords.length]
y = new double[coords.length]
for (i = 0; i < coords.length; i++) {
x[i] = coords[i].getX()
y[i] = coords[i].getY()
}
// Transform x and y into binary image, via ImgLib2-ROI
import net.imglib2.FinalInterval
import net.imglib2.roi.geom.GeomMasks
import net.imglib2.roi.Masks
import net.imglib2.view.Views
polygonRoi = GeomMasks.polygon2D(x,y)
mask = Masks.toRealRandomAccessibleRealInterval(polygonRoi)
result = Views.interval(Views.raster(mask), new FinalInterval([0, 0] as long[], [300, 300] as long[]))
Instead of the FinalInterval at the end, you can use your original Img (which is an Interval as well).
As you can see, the original bitmasks on the left get translated to text just fine. But when I reconvert them to bitmasks again as you can see on the right, they are displayed a little differently. It doesn’t matter too much, because I still can use the GroupBy node to assemble an actual labeling like this:
But it makes me wonder if it has some performance implications for larger images. It seemed as if the GroupBy node would execute faster using the original bitmasks compared to using the reconstructed “bigger” ones.
The bit masks generated by the Segment Features node (in the Bitmask column) are just as large as necessary to contain the bounding box of each segment. The positional information is saved in the image offset (that can be read/set by the metadata-handling nodes).
KNIME Image Processing (KNIP) uses the min value of an ImgLib2 Interval that can be at any offset.
Side note:
If I remember correctly, KNIP uses this in a slightly wrong way, since it allows a non-zero min for Img (instead of RandomAccessibleInterval) objects, despite the javadoc of Img explicitly stating:
You can probably do the same in your plugin by subtracting the minimum coordinates of each polygon from the interval bounds when creating the mask image, and then setting the min accordingly. I didn’t try it yet, though…
If you think it’s of general use and should be part of fmi-ij2-plugins, I’m happy to accept pull requests!
Definitely have done this from within a Java Snippet, but I also don’t see why it shouldn’t work in a Command.
Interesting observation, @imagejan. I guess it boils down to KNIP being very “relaxed” (not necessarily in a good way) about RAI vs Img, in general. Does native KNIP ever create Imgs with non-0 min or is this only an issue with the ImageJ2 integration?
I would argue that it is of general interest. Plus, it would really be a pity if this got lost on a branch…
As far as I can tell, this is pretty omnipresent in the offset-handling of KNIP. For example, this method wouldn’t make any sense at all if Img objects would be treated to have always zero-min (as defined in the Img javadoc):
Here we go, my pull request! Please review and let me know if you want me to change anything. I wasn’t that sure about what to put in the initial javadoc at the beginning of the class files, but I hope what I did is fine. Thanks for encouraging me and thanks for all the help! Create plug-ins to convert well-known text to bitmasks and vice versa #14
