Adding Maven dependencies to KNIME node plugin

I am trying to add some none-Eclipse p2 dependencies (specifically, dependencies which are available on Maven Central) to a custom node plugin. I was searching this again to see if there is a better alternative than the 2 ‘legacy’ or ‘pure Eclipse/OSGi’ approaches of:

  • Download the jar and put it in a /lib folder in the plugin using it and add to class path
  • As described in the node guidelines ‘If it might be useful, wrap it in its own plugin for reuse’

(I’ve never done the second as never been entirely sure on the precise details of doing so!)

So, I did a bit more searching and came across this useful anser from @gab1one

Continuing the discussion from Importing Maven Dependencies into KNIME Nodes:

My question is though - can I use that as part of a multi-module build to ‘just make those available to the other modules’ (and if so, how do I do that?!) or do I have to have a separate project to build and publish the update site somewhere as in the README for GitHub - reficio/p2-maven-plugin: Maven3 plugin that automates the third-party dependency management for Eclipse RCP ?

Thanks

Steve

Hi Steve,

The pattern that we use to retrieve dependencies for this use case can be seen here:

This is a bundle that contains a bunch of dependencies that are placed in the libs folder and retrieved via a fetch_jars project during build time:

best,
Gabriel

2 Likes

Thanks Gabriel - that looks really useful.

Does any magic happen with the libraries_license/licenses.csv file or is that simply a way of listing the licenses for the bundled jars?

Steve

Yes that is just for keeping track of the licenses

1 Like

Hi Gabriel, funny that I am looking in this moment also for such a solution, because I need to add a library to our Novartis plugin, which brings in alone 24 dependent jar files - it would be real pain adding them manually. I looked at your pom.xml solution and I think I have that already since a couple of years done like this myself. BUT: Fetching the JARs is only half of the work. A very annoying part is to add/modify all specific JAR file names (which usually include a version number that change over time) in the MANIFEST.MF file as well as in the build.properties file. Is there a solution, that keeps them in sync with whatever is located in the lib folder (after the automated fetching?). It would have to be a task that runs after the copying process. If not, I am wondering if it would be worth writing a little Maven plugin to do exactly this. Ideas?

1 Like

I only explored this so far twice, with respectively a single jar (+ source jar) and 2 jars (no sources), but I was already also wondering about it. Maybe a KNIME workflow as an interim to a maven plugin?!

Hi @manuelschwarze and @s.roughley

We actually have an automated bin.includes / MANIFEST.MF update process in a project where we include a lot of jars: Take a look here knime-bigdata/org.knime.bigdata.spark.local/libs/fetch_jars/pom.xml at master · knime/knime-bigdata · GitHub

best,
Gabriel

Hi @s.roughley and @gab1one and everybody interested,

There is now an even better way available. Last week I have written and open-sourced a small Maven plugin that updates properly the Eclipse based MANIFEST.MD and build.properties files which are referencing JAR libraries. So it got very easy now to manage via a Maven pom.xml placed in a dedicated library folder all necessary dependencies and to keep the Eclipse-based files in sync.

I am using this myself now for RDKit nodes (there is only one direct dependency to the OPSIN library, which brings a few other dependencies with it), but also for our Novartis Biomedical Research extension, which has a lot of dependencies.

More information you can find directly in the open-source repository of the KNIME RDKit Extension:

The main difference to @gab1one 's approach is (as far as I understand it), that with my Maven plugin you can easily combine Maven and manually handled JAR files. You simply have to separate them into two different library sub folders. Another benefit is that it is much simpler to configure.

I am using the Maven plugin only manually as developer, and then I am checking in results (JAR files, MANIFEST.MD, build.properties), and those results are used when building the final KNIME extension with Tyco. However, if desired I think one could also simply integrate this into any Tyco build process and just generate the JAR files and file changes on the fly without keeping the JAR files in a git repository. This could make sense, if you have tons of JAR file dependencies.

I hope this will be useful for some other KNIME Community Developers, at least for me it is already. :slight_smile:

Kind regards,
Manuel

2 Likes