Building KNIME with Buckminster

Hi all,

I have looking into Buckminster and Jenkins in the last weeks and tried to follow the tutorial from Thorsten, https://www.knime.com/blog/creating-knime-update-sites-with-buckminster. While I managed to build the example from the tutorial I could not build my own nodes. I guess some extra configuration files are needed and I decided to post my installation as it may help others.

First I installed the Buckminster plugins from http://download.eclipse.org/tools/buckminster/updates-4.5. I could not find update sites for newer Eclipse releases.

  • Buckminster Core
  • Buckminster - PDE Support
  • Buckmnster - Git Support

The first problem with the tutorial is that the required SVN plugins are not supported anymore. Or at least the versions needed by Buckminster. I solved it by copying the example projects to a public Git repository and using the Git reader type in resource map file. The Git repository can be accessed at https://gitlab.bfr.berlin/miguel/nodes4knime and only contains the three example projects:

  • my.example.feature
  • my.example.plugin
  • my.example.update

The command query file is left unchanged. It even has the SVN properties which are not used now.

<?xml version="1.0" encoding="UTF-8"?>
  <cq:componentQuery xmlns:cq="http://www.eclipse.org/buckminster/CQuery-1.0" resourceMap="knime.rmap">
  <cq:rootRequest name="my.example.update"/>
  <cq:property key="svn.password" value="knime"/>
  <cq:property key="svn.user" value="anonymous"/>
  <cq:advisorNode namePattern="^my\.example\..+" sourceLevel="DESIRE"/>
</cq:componentQuery>

In the resource map file all that changes is the search path that now points to my Git repository instead of the KNIME community one.

<rm:searchPath name="my-stuff">
  <rm:provider readerType="git" componentTypes="buckminster,osgi.bundle,eclipse.feature">
    <rm:uri format="{0}/nodes4knime,{1}">
      <bc:propertyRef key="workspace.root" />
      <bc:propertyRef key="buckminster.component" />
    </rm:uri>
    <rm:property key="git.remote.uri" value="https://gitlab.bfr.berlin/miguel/nodes4knime.git" />
    <rm:property key="git.remote.name" value="nodes4knime" />
  </rm:provider>
</rm:searchPath>

With these changes now I can resolve and materialize which seems to work as it makes a builds in /tmp.

The problems come with my extension node plugins at https://github.com/SiLeBAT/FSK-Lab. The resource map seems to work and they are retrieved but Buckminster fails to build them. Maybe I skipped some Buckminster configuration file. How should I proceed? I am attaching the command query and resource map files for my plugins.

Thanks,
Miguel

fsklab.cquery.xml (270 Bytes)
fsklab.rmap.xml (1.4 KB)

What does “fails to build” exactly mean?

Hi Thorsten,

Buckminster pulls the projects but then throws the following errors:

Cloning remote repository https://github.com/miguelalba/FSK-Lab.git into D:\buckminsterWS\FSK-Lab
ERROR [0002] : No suitable provider for component de.bund.br.knime.nodes.util:osgi.bundle was found in resourceMap file:/D:/bm-example/fsklab.rmap
ERROR [0002] : No suitable provider for component de.bund.br.knime.nodes.util:osgi.bundle was found in searchPath knime-stuff
ERROR [0002] : Rejecting provider p2(http://update.knime.org/analytics-platform/3.4[http://update.knime.org/analytics-platform/3.4]): No component match was found
Errors and Warnings
E [0002] : No suitable provider for component de.bund.br.knime.nodes.util:osgi.bundle was found in resourceMap file:/D:/bm-example/fsklab.rmap
E [0002] : No suitable provider for component de.bund.br.knime.nodes.util:osgi.bundle was found in searchPath knime-stuff
E [0002] : Rejecting provider p2(http://update.knime.org/analytics-platform/3.4[http://update.knime.org/analytics-platform/3.4]): No component match was found
TAG-ID 0002 = Query for de.bund.bfr.knime.fsklab.feature, path: de.bund.bfr.knime.fsklab.feature:eclipse.feature$1.0.0.qualifier

BTW, something weird I noticed is that next to my plugins the example projects were also downloaded but they are not included in the resource map.

Apart from your rmap not being correct or the project not being a plug-in I cannot see any other potential problems.

One suggestion, looking at our Jenkins job and rmap is that we check out the source code for our nodes in to the Jenkins job workspace, and then use a local provider rather than the git/svn providers, e.g.:

<rm:locator
	pattern="^com\.vernalis\.knime\.internal(\..+)\.feature"
	searchPathRef="feature"
	failOnError="false" />
<rm:locator
	pattern="^com\.vernalis\.knime\.internal(\..+)"
	searchPathRef="local"
	failOnError="false" />

and

<rm:searchPath name="local">
	<rm:provider
		componentTypes="osgi.bundle,eclipse.feature"
		readerType="local">
		<rm:uri format="{0}/sources/{1}/">
			<bc:propertyRef key="workspace.root" />
			<bc:propertyRef key="buckminster.component" />
		</rm:uri>
	</rm:provider>
</rm:searchPath>

<rm:searchPath name="feature">
	<rm:provider
		componentTypes="osgi.bundle,eclipse.feature"
		readerType="local">
		<rm:uri format="{0}/sources/{1}/">
			<bc:propertyRef key="workspace.root" />
			<bc:propertyRef key="buckminster.component" />
		</rm:uri>
	</rm:provider>
</rm:searchPath>

Steve

1 Like

When I took over our knime package from a colleague I was also confronted with buckminster and found it very hard to understand. I then switched to maven / tycho, maybe it’s worth to have a look at it: https://www.eclipse.org/tycho/

Since I am using eclipse (as well as knime itself) as ide, I also find tycho much better integrated into eclipse, which makes depeloping much easier for me.

Best, Frank

Thank you all. Sorry for answering so late. I was on holidays leave.

@thor the project de.bund.bfr.knime.fsklab.feature is not a plugin but a feature project like in the example from the tutorial, my.example.update. In this example project there are also two configuration files: bm.properties and buckminster.json. Maybe there are needed in my project?

@Vernalis thanks for your suggestion! I have seen it in other Buckminster configurations but my idea was to use the mutable property of the provider. If I am not wrong, this only pulls the changes and not the whole repository on each build.

@sonnenburg currently I a looking into Tycho but so far I have not been able to debug the Tycho build with a target as an RCP application. Following the Vogella tutorial at http://www.vogella.com/tutorials/EclipseTycho/article.html#removing-compiler-warning-messages-from-the-build I created a build for an update site but I ignore how to launch it from Eclipse. The only idea I have so far is to create a product configuration.

Best,
Miguel

Hi @miguelalba,
did you have a look at https://github.com/3D-e-Chem/tycho-knime-node-archetype yet? It is a community maintained maven prototype for developing KNIME nodes. It should be relatively easy to adapt it to your needs.
best,
Gabriel

Hi @gab1one

yes I know that project. I started with it but I found it confusing since you get the whole structure at once. Following the vogella tutorial I created the same RCP project structure with the example KNIME node and uploaded to GitHub at https://github.com/miguelalba/my.example. Pretty glad with Tycho so far:

  • Dependencies are retrieved from a KNIME target.
  • Only tests I included so far are unittests. In the maven archetype they are including test workflows which I will look into.
  • CI with Travis. The single example node is taking 2 minutes to build and half a minute when caching the dependencies. It will be interesting to see with many extension nodes and test workflows.
  • I failed to launch as RCP application from a product project but succeded from the bundles project (nodes project).
  • I ignore how to use 3rd libraries as Maven dependencies in Tycho. There seems to be some workarounds.

Best,
Miguel