New Image Node

KNIME Extensions Image Processing
#1

Hello everyone,

I think the KNIME workflow model is well suited to image processing and I'd like to use it with custom algorithms. I can program in Java (experience with ImageJ) but I cannot figure out how to create new image processing nodes in KNIME. I found a tutorial about creating new nodes and I can create a sample node (NumericBinner example in the tutorial).

Being new to KNIME, I have some basic questions to which I couldn't find the answer (it would be good to have a very basic introduction page).

I'd imagine the node creation process as follows: I create a .jar with the node code in the KNIME SDK, I load the .jar in KNIME Desktop and start using the node in workflows. Apparently the system is way more complicated than that.

What I understand about the process: I launch the KNIME SDK (Eclipse), create a new node project, write my code. Then I run KNIME from the SDK with Run->Run, and I use the node. But how can I use the node in other KNIME instances, e.g. how can I send a node to someone else? What do I need to do and what does the recipient need to do (in terms of basic menu steps)?

Do I have to use the Eclipse software installation functionality for this?

About Image Processing: Is there a sample image processing node that I could start with to create my own? I tried to look in the sources and copy a node with a different name for a start, but I cannot find where the code of the nodes are. I looked in org.knime.knip.base.source and org.knime.knip.core.source.

 

Thanks.

 

#2

Hello Sarandi,

this sounds great. Of course we will support you wherever we can.

1. About KNIME Development/Node Deployment:

As you already pointed out correctly, you develop your node in using the KNIME SDK and then use "Debug->Run" to test/debug your node. This is very handy, as thanks to hot-code replacement you can even debug your code while running your KNIME instance.

If you want to use exactly this code on another machine (deploying the node), you have several options. The easiest option is, that you export your node  using the "Export Wizard" in the plugin.xml tab "Overview". This wizards creates a new .jar depending on the settings of your build.properties file in the plugin (see standard Eclipse Plugin development guide-lines for that). This jar you can then copy paste in the drop-in folder of the KNIME installation in which you want to use your node. I completely agree, that this is not very handy and actually not very convenient.

Therefore there are more sophisticated ways to deploy your nodes on other machines. Depending of whether you want to make your code available for everybody, you could use the infrastructure of community-contributions, provided by KNIME. KNIME provides so called update-sites for projects, such that you can easily install your set of nodes (i.e. your plugin) on other machines. But this is only worth it, if your nodes are of general interest. Contact for Community Contributions: thorsten.meinl@knime.com

Anyway, you can also create your own update-sites. See eclipse documention of how to create update-sites. Howevery, as soon as you really want to deploy your code we can discuss all these options again and help you there of course.

2. About Development of KNIME Image Processing Nodes:

First, we heavily use ImgLib2 in the background to process images (imglib2.net). It's really worth learnin ImgLib2 as also the new ImageJ2 will completely base on ImgLib2. ImgLib2 at the beginning is not the most intuitive framework, but from our experience the learning rate is very high and you will achieve nice results after a few hours of trying. On imglib2.net there are very nice tutorials to learn about imglib2.

Second, all our nodes / code / plugins are available on https://github.com/knime-ip/knip. To develop KNIME Image Processing Nodes I would suggest the following steps:

Clone the repository from https://github.com/knime-ip/knip (see https://help.github.com/) for help. After cloning (this means creating a local copy) of the repositories you will have to add them to your KNIME SDK. Just use Projects -> Import existing projects to KNIME
and import all the KNIP related projects. See: https://github.com/knime-ip/knip Package Organization in the readme for details. In your plugin.xml I would now add a dependency (Tab: Dependencies) to org.knime.knip.core and org.knime.knip.base. Now you can use ImgLib2 + our KNIME Image Processing stuff in your plugin. See org.knime.knip.exampleplugin as an example. We also have more abstract nodemodels which provide even more functionality than the ValueToCellNodeModel which you find in the example (e.g. ImgPlusToImgPlusNodeModel), but you can find examples for that in our org.knime.knip.base plugin (e.g. InvertNodeModel).

I think for now, this is a good starting point. Again: if you face _any_  problems or if you have any questions, we would be happy to help!

Good luck ;-))

Christian

 

 

#3

Thank you very much for the super-quick response!

I'll try your suggestions. But first I'd like to ask some quick questions to see if KNIME is really right for me.

- I tried building a simple workflow, but got stuck at the beginning when I looked for a node to convert RGB to grayscale. Apparently this very basic operation has no node.

- Looking into the source I found org.knime.knip.core.ops.transform.HoughLine, but I found no Hough Line Transform node. Am I overlooking something?

As a first exercise (to get familiar with things) I'd like to make a simple workflow: Load image, convert to grayscale, binarize, find Hough lines, take the most voted line and write its parameters to a file. Then as a more complicated version do this in bulk: Iterate over all files in a folder and then at the end generate a single CSV file containing in each line the image filename and then the corresponding most voted Hough line parameters.

It would also be nice to be able to inspect the Hough space (render the votes), but do this only on request, so it wouldn't waste time when it's not needed. Maybe this could be another output port of the Hough Lines node. Can a node be lazy with respect to an output port? I mean could the Hough Lines node see if it has any consumer nodes on its "Hough space image" port and generate it only if necessary?

Is KNIME Image Processing suitable for such things?

#4

- RGB to GreyScale:

Actually we have various options to do so: For example you can use the projector the project in the channel dimension or you could use the splitter node followed to the image calculator to define your own formula. Or you can run an arbitrary ImageJ1 Macro (ImageJ Macro Node) which does the job for you?

Additionally you always have the option to write your own node, see http://tech.knime.org/forum/knime-image-processing/color-extraction.

- Hough-Lines:

We don't have a Hough-Line Node yet, as we couldn't find a way to generally express the output of such a node. 

About your project: This sounds pretty do-able, but I think you have to write the Hough-Node for that (of course you can use the imglib2 operations to do so, this sounds pretty straightforward and is a nice exercise I guess). So your first workflow would look like this:

Image Reader -> Projector -> Global Thresholder (for binarization) -> Export Hough Lines (your node which internally uses Hough from imglib2 internally and already spits out a table with all the lines + rank) -> CSV Writer. The Bulk Version is then no problem anymore, you just use the List Files -> Image Reader -> Chunk loop start -> Workflow see above -> Chunk Loop End. Actually you can also leave the chunk loop start/end away if your hough-node is capable of handling multiple image in a column (depends on your output table, see below).

About the Lazy Ports: Sure. As long as you have the information how an image is rendered, your can render it only on demand. But this would be more like a "View" (right click on node and then View: Hough-Space, here you can use our image viewer stuff, we can help you with that) on your node.

Hints for your implementation: You can't use ValueToCellNodeModel for your Hough-Node. Reason: ValueToCellNodeModel (and childs) assume, that you always want to process one image after the other of a table column and spit out another column where the results are put in one after the other. For your RGB -> GrayScale Node this makes sense (take one image, process it, write it back). In the case of Hough you want to input images from one column an output a table in the form (SourceImg, LineParameters, Score/Rank). Afterwards you would like to use some filtering (e.g. GroupBy) so determine the best line per file...).

For such kind of analysis you can have a look e.g. at the ImgPropertiesNodeFactory. Imgs are the input, output is a table.

Does this help?

Christian

PS: There is a webinar about KNIME this evening, maybe you can still register: http://www.knime.org/webinar-knip-march-2014

PPS: Chosing KNIME is always right ;-)