Manipulating actual image pixels?

KNIME Extensions Image Processing
#1

Hello,

I am working on analysing some x-rays and specifically the first step I need to to is normalise all the x-rays, i.e. correct the rotation of every x-ray.

Now, I already have done this in Java using matrix manipulation - I essentially converted this matlab code: http://academic.thydzik.com/computer-vision-412/lab-2.htm into Java using UJMP - a Java matrix library.

I am now trying to implement this algorithm using KNIME and it seems like a really good tool and finding the first things I need is dead easy, i.e. finding the centroid etc. However, I now need to find the central axis (or more specifically I need thetamin, looking at the link) and the issue is the fact that I need to find the denominator which is calculated using the actual image pixels.

Now, is there a way to access the pixels, do some operations on them using R or Java and then return the values? In any case, maybe there is a better way of doing this using KNIME?

Thanks,

Paulius.

#2

Oh! I found the ImageBee plugin which has a node called "ImageToTable", I guess this is exactly what I need!

Unless you can offer a better way of doing what I am suppose to do?

#3

Hello Paulius,

I would write an ImageJ2 Plugin which then can be loaded into KNIME as a KNIME node.

You find a very nice example how this may look like right here:

https://github.com/knime-ip/knip-imagej2/blob/master/org.knime.knip.imagej2.fragmentdemo/src/MySimpleThreshold.java

This plugin can then be installed into KNIME (see: http://tech.knime.org/community/imagej) and will appear as a node KNIME node!!

Inside the ImageJ2 Plugin, you can access the pixels using ImgLib2 (imglib2.net provides very nice examples/tutorials).

Anyway, we are currently working on a JavaScripting node for ImageProcessing, but this one is under heavy development and will be available only in 2 or 3 months.

Hope this helps,

Christian

 

 

 

#4

Hey Chris, sorry for the slow response. Thanks a lot for you links, however, I have a question (problem) about the plugin ImageBee (http://tech.knime.org/imagebee-analysing-imaging-data-from-the-honeybee-brain).

As I said before it seems like it has the correct node that I need (even though I don't really do any analysis related to honeybees) - the ImageToTable node. However, there seems to be a bug (?) with that node - it seems only to accept image with multiple channels, so the following error occurs when I tried to execute it with a DICOM file (which has only one channel):

Execute failed: ("ArrayIndexOutOfBoundsException"): 2

Look at the source for the plugin I found the following line (package org.knime.neuro.misc.imagetotable, class ReadData):

frames+=img.dimension(2)

Which I am pretty sure is the culprit. Is there any chance for this to be fixed to accept single channel images?

Thanks,

Paulius

#5

Hi Paulius,

I'm the honeybee plugin is not mainted by us (KNIP Team). It's provided by someone else, so maybe you can use the contact information from http://tech.knime.org/imagebee-analysing-imaging-data-from-the-honeybee-brain to contact the plugin author.

Anyway, I really doubt that you have to use the node. There are only very, very rare cases when you need to transform an image to a table. What you want to do, is to manipulate the pixel values, right? For example, as an example, inverting the pixel values or getting some information like average intensity out of the image, correct?

So in this case you have to implement your own node (as our scripting node is not ready, yet) and then use ImgLib2 to manipulate the object on a _Java_ level.

Writing an image processing node is very easy, see my links above. You only have to implement a ImageJ2 Plugin, install it and the node will automatically appear.

If you need any asstistance setting up the KNIME SDK for developing nodes, let me know. I'm happy to help.

Best,
Christian

 

#6

Thanks Christian. I guess you are right and this makes little since - there must be a faster and better way of doing this and the one you're suggesting probably makes much more sense in KNIME.

My actual goal, in terms of end-product, at this phase is very simple: given an X-ray (like this one: https://www.dropbox.com/s/5yf4v1eoekvar2j/big.jpg) and straighten it so the mouse head faces upwards as accuractly as possible. The algorithm I linked to in the original post of this thread (http://academic.thydzik.com/computer-vision-412/lab-2.htm) essentially gives me the angle of the mouse, by which I can simply rotate to get the mouse to the upright position.

What it actually calculates is the central axis using the centroid (centre of mass). Maybe you know of any better ways of doing this? If something is unclear please tell me, I'll do my best to explain.

Also, regarding making the ImageJ2 plugin - I have cloned the repo in which you got the example, the issue is - how do I set it up with all the depedencies? There are no pom files or anything for Maven to work with..

Thanks,

Paulius.

#7

Hi Paulius,

I will write a complete guide how you get your stuff running or how one can develop KNIME Image Processing Plugins (with or without ImageJ2) using the KNIME SDK. I try to write a complete guide( maybe I can put it into the Wiki later), so just ignore the parts you already know ;-)

1. Download and Install KNIME SDK

Under Windows and MAC you can use the installer, under linux just unpack th e file and run eclipse. The KNIME SDK is actually an Eclipse with all the required KNIME plugins.

2. Now you have two options:

a.) Clone our git repositories: http://github.com/knime-ip/knip and http://github.com/knime-ip/knip-imagej2. Afterwards import the Java-Projects into the KNIME SDK (File -> Import -> General -> Existing Project Into Workspace). Be careful, don't check the option "copy content to workspace" as you then won't be able to see the updates of Git reflected. Note: Of course you can also use your own Eclipse Git integration (e.g. EGit).

b.) Use the update-sites of the KNIME SDK to install KNIP and KNIP ImageJ2.

Whats the difference between (a) and (b)? Well, obviously using option (b) you won't see the code of KNIP or KNIP-IJ2, but maybe you don't need it. Additionally, with option (b) you don't need to bother with git (even if it is very easy to use).

3. Start KNIME SDK

I prefer to use the debug version of KNIME as you can set breakpoints to debug your nodes etc. Therefore you left-click on the arrow right to the small "Debug Bug" in the toolbar over your projects / code window. Then click on "Debug Configurations". Create a new Eclipse Launch configuration. The only thing you need to configure are the VM parameters in the tab tab arguments:

Depending on how much memory your machine has, you should set the -Xmx parameter. Additionally you have to set a parameter -XX:MaxPermSize=192m.

My vm argument line looks as follows: -Xms40m -Xmx6G -XX:MaxPermSize=192m

After you started KNIME from the KNIME SDK, you have to switch to the KNIME perspective in the newly started KNIME (this setting will be kept in the future).

What did you just do? Well, you started KNIME (which is an eclipse application) from your KNIME SDK. The loaded KNIME contains all plugins which are available in your project list + all plugins which are installed in the KNIME SDK.

4. What are these org.knime.knip.core etc projects?
They are not maven plugins, they are Eclipse Plugins. If you want to add additional dependencies you can do so by linking on another eclipse project or adding a new libary to the classpath. You always do that in the plugin.xml and never via any other mechanism (like build-path configuration of eclipse). You add new eclipse projects as dependencies in the dependency tab of the plugin.xml, new .jars in the Runtime tab under classpath.

5. How do I create my own plugin?

Honestly, just copy and paste an existing one. Thats the most convenient way to create a plugin.

----

Now related to your questions:

* You also should clone KNIP repository. If you cloned both above mentoined repos you shouldn't have any problems with dependencies anymore in the existing projects.

* The problem looks pretty doable. Actually, it seems to be a very nice use-case for learning how to write IJ2 plugins / KNIME nodes. You would write an ImageJ2 Plugin which requires a Labeling as an Input and maybe some parameters specifying your axis as an output. Before that, your workflow would first smooth the image a bit (Quantil Filter), then using a simple thresholding algorithm (looks suitable concerning your images), followed by a Connected Component Analysis. Then your node will follow.

A labeling is nothing else as a representation of a segmentation, but I think you can see that when you play around with the example workflows. :-)

I hope this somehow helps! If not, let me know, then I can update the Guide and help you at the same time ;-))

Christian

#8

Thanks a lot Christian, that certainly helps.

I really this is basically my last struggle and I will be able to work on from this alone.

There are two things which are not enitrely clear to me:

  1. I got the KNIME SDK working, I managed to create a Node project, everything seems fine, however, I am having trouble accessing the ImgLib2 library in the node project. I am sure this is extremely simple as I have looked at code for other nodes and many use the imglib2 library just fine. How can I do this?
  2. Is there any way to quickly use the node inside the KNIME SDK? As of now, from what I gathered, you have to do Export -> Plugin (etc.), get the jar, put it in the dropins folder of KNIME and then you are able to actually put it as a node. Now, this all works fine, however, this seems to be very slow way of testing. Exporting it every time wastes a lot of time. Is there any way of doing this quickly from within the SDK?

Thanks again,

Paulius

edit:

I got another question, I had a look at developing an ImageJ2 plugin and I encountered a problem there as well:

I have imported all the projects into Eclipse (http://i.imgur.com/qlsCVd9.png), despite the errors it seems ok beside one issue - how do I actually get this it to KNIME? I am just trying the "fragmentdemo" one, specifically the "MyThresholder" plugin.

I export a simple JAR (guess this is the issue here) then go to KNIME -> Preferences -> KNIME -> Image Processing .. -> ImageJ2 Plugin Installation, select the JAR and restart KNIME, but for some reason the node doesn't appear in the list?

As I said I am pretty sure the issue is how I "export" the plugin, however, I couldn't find anywhere how do I actually do that so it would work with KNIME. Any chance you could explain that?

Thanks again.

#9

Hi Paulius,

here the answer to your questions:

1. You of course can create an own Node Project using the KNIME wizard, then you can add dependencies to your project using the plugin.xml file. Especially the tabs dependencies / runtime are interesting (see my post above). You can add the project org.knime.knip.core to as a dependency to your project and then all libraries which are exported by this project are available in your project (especially imglib2 in this case).

2. You don't really have to do that ;-) Again, as mentoined above, just start a new KNIME out of the KNIME SDK using the debug configuration (see here
http://tech.knime.org/test-your-node and the screenshot attached to the workflow). Then all plugins which are available as java projects will be loaded as well. Nothing to export etc. You can even set break-points in your code for debugging or change the code while the second knime is running (as long as you don't manipulate any function names, add functions or member variables in general).

If you want to develop an ImageJ Plugin I would just copy the example project and modify it, as you have to do one more additional setting to your project, which is due to ImageJ2. You have to add a AnnotationPreprocessor (right-click on project and have a look at builders -> EclipseHelper in the ImageJ project). Anyway, if you copy the existing project, you are fine.

3. So for developing you really can stay in the KNIME - SDK, you don't have to export _anything_! If you have a project ready to be deployed (and used by others) you have some options: a.) You export the plugin and copy/paste it or b.) you create your own update-site which others can use (which is also not too hard). Anyway, I think we come back to this point after you have some code to deploy :-))

Does it help?

 

 

 

#10

That's great! Got it all to work now, thanks a lot! I should be fine from now on - now it will be an issue of learning how to use ImgLib2 and actually solving my problem.

Thanks again.

#11

I'm really sorry for being this annoying but there seems to be a problem - I test the node just as you said (and shown in your link), but there seems to be a problem in the KNIME/Eclipse that launches - if I put an Image Reader node (either the experimental one or the "normal" one) and double click to configure it, KNIME/Eclipse stops responding..

The error message seems random and even more the whole behaviour of this seems random.

Now I'm getting

Error while logging event loop exception: Exception in thread "[Timer] - Main Queue Handler"

But it just worked a couple of moments ago just not to work again, sometimes I also get a real error but it decided on stop printing that.

Any ideas?

Cheers,

Paulius

edit:

Got a different error:

Exception in thread "[Timer] - Main Queue Handler" Error while logging event loop exception:
ERROR     main NodeContainerEditPart     The dialog pane for node 'IJ Image Reader (2D) - experimental 0:2' has thrown a 'OutOfMemoryError'. That is most likely an implementation error.
Error while logging event loop exception:

!ENTRY org.eclipse.osgi 4 0 2014-08-22 14:00:56.983
!MESSAGE Application error
Exception in thread "State Saver"

#12

Hi Paulius,

no problem at all. You can ask as much as you want ;-) We will get this working!!

Can you post your VM Argument line? I think there might be the problem.

 

#13

Hey Christian, again, thanks a lot for all of the help!

I am using "-ea -Xmx1024m". Also, it seems to work just fine if I run it normally, i.e. if I open regular KNIME. It only breaks when I tried to run the reader after opening KNIME/Eclipse using the "run" button in KNIME SDK.

Hope this is clear.

#14

I would (a) increase your memory a bit if you can (like 2G or 4G) and b.) did you set XX:MaxPermSize=192m?

 

#15

Seems to work fine now, thanks a lot! Hopefully all will be well now, cheers!

#16

great! if not, let me know. Again: I really want to help you here, I think its worth it.

Christian