Using Knime as external class library

KNIME Analytics Platform
1

Hello Knime Workgroup

I am currently analysing a huge set of data using knime. The data is structured into subsets with identical properties but changing characteristics. I also build a knime (v 1.2.1, developer version) workflow to analyse and visualise one of these subsets and the result was very good. The problem is the number of sets I like to analyse using knime. I do not want to execute the workflow for each of these sets, it is just too much handwork and not applicable for an automated process.

I read below, that it is not possible to iterate a knime workflow as such, unless I use the knime batch process, which I don't want because I have to pre-process and post-process the date I which to send to knime. Storing everything in a flie or database is too much of an afford because of its one time use only (i just did it to test the knime on my data). Therefore, I would like to use the knime functionality as external class library, to invoke knime nodes from within my program. Unfortunately, using the jar-libraries of the eclipse plugins seem not to be working for this.

The following example leads to:

import org.knime.base.node.mine.cluster.fuzzycmeans.*;

...

FuzzyClusterNodeModel clusterModel = new FuzzyClusterNodeModel();

leads to the following exception:

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/log4j/spi/Filter
at org.knime.core.node.NodeModel.(NodeModel.java:110)
at org.knime.base.node.mine.cluster.fuzzycmeans.FuzzyClusterNodeModel.(FuzzyClusterNodeModel.java:206)

The same problem occurs if I try to initialise a factory class.

I would be thankful for any help,
Roland

2

Hi Roland,

In principle, it should be possible to use the (core) KNIME as external libraries. However, there are dependencies to some Eclipse-plugins like e.g. the XML parser or the plugin handling stuff that ensures that the plugin activators are called before the plugin is loaded. The XML parser issue can be solved by specifying your "own" XML parser (see e.g. http://java.sun.com/javase/6/docs/api/javax/xml/parsers/DocumentBuilderFactory.html#newInstance()) on how to do this. You may also work around the code that is executed inside the plugin activator(s) by emulating what's going on there by yourself.
In your specific case it seems you also forgot to include the log4.jar into the classpath, which is part of the org.knime.core-plugin.

Regards,

Thorsten

3

Thank you, the log4.jar was indeed missing. Knime is an interesting Project, thank you very much again, also for your fast response!

Greetings,
Roland

4

What is the current "best practice" on setting up KNIME in batch mode or be able to trigger multiple dataflows (for example in overnight processing).

Thanks

Iqbal

5

Iqbal wrote:
What is the current "best practice" on setting up KNIME in batch mode or be able to trigger multiple dataflows (for example in overnight processing).

Just use the BatchExecutor :D Here is how it works: http://www.knime.org/faq.html#g12

Regards,

Thorsten