Size of the Mountain ???

Dear All

I am wondering if anybody can point me in the direction of any information / tutorials & self learning material which will help me to get a better understanding of how to use Knimes Image processing capabilities and deploy it to clouds/clusters i.e. amazon cloud, HAdoop, Spark...........i have a grasp of the basics of programming in C++, MAtlab, R and Fortran and have an idea of what Java and python can do but need to get an idea of where i need to focus and what i need to learn.....At the moment i seems like a huge mountain to climb, however perhaps somebody can provide a less difficult route to the top....a guide for dummies if you will

Regards, DJ

Dear DJ,

the mountain is not as high as it seems. Scaling up KNIME and KNIME Image Processing in particular can be done in several ways and we are currently working on reducing the height of the mountain more and more.

1. Calling KNIME in batch mode headless and deploy it on cluster architectures like SGE, LSF, SLURM etc.:

https://tech.knime.org/faq#q12.

2. Deploying KNIME using the SunGridEngine Cluster Execution

https://www.knime.org/cluster-execution

We are currently working on more connectors for DRMAA cluster architectures, e.g. Unicore (open-source, https://www.unicore.eu/) and SLURM.

3. Running KNIME Image Processing on Spark:

This is something which is under development. Our plan is a seamless integration of KNIME Image Processing and Spark (similar to the Cluster Extension). A lot of classical machine learning or data-mining algorithms are already supported with https://www.knime.org/knime-big-data-extensions and can then be used in combination with KNIME Image Processing.

Learning more about KNIME Image Processing itself:

* https://tech.knime.org/community/image-processing there you find a link to some tutorials / lectures and also instructions how to access the example server where a lot of example workflows are available.

Let me know if you need more details on a certain topics.

I hope this helps,


Christian